Abstract:
A method of utilizing metadata stored in a computer-readable medium to assist in the conversion of an audio stream to a text stream. The method compares personally identifiable data, such as a user&#39;s electronic address book and/or Caller/Recipient ID information (in the case of processing voice mail to text), to the n-best results generated by a speech recognition engine for each word that is output by the engine. A goal of this comparison is to correct a possible misrecognition of a spoken proper noun such as a name or company with its proper textual form or a spoken phone number to correctly formatted phone number with Arabic numerals to improve the overall accuracy of the output of the voice recognition system.

Description:
CROSS-REFERENCE TO RELATED APPLICATIONS 
       [0001]    The present application is a nonprovisional patent application of, and claims priority under 35 U.S.C. §119(e) to U.S. provisional patent application Ser. No. 61/041,220, filed Mar. 31, 2008 and titled “USE OF METADATA TO POST PROCESS SPEECH RECOGNITION OUTPUT,” and is hereby incorporated herein by reference in its entirety. 
         [0002]    Additionally, U.S. nonprovisional patent application Ser. No. 11/697,074, filed Apr. 5, 2007 and published as U.S. Patent Application Publication No. US 2007/0239837, is incorporated herein by reference, and each of the following patent applications, and any corresponding patent application publications thereof, are incorporated herein by reference:
       (1) U.S. nonprovisional patent application Ser. No. 12/197,213, filed Aug. 22, 2008 and titled “CONTINUOUS SPEECH TRANSCRIPTION PERFORMANCE INDICATION;”   (2) U.S. nonprovisional patent application Ser. No. 12/197,227, filed Aug. 22, 2008 and titled “TRANSCRIBING AND MATCHING MOBILE DEVICE UTTERANCES TO KEYWORDS TAKEN FROM MOBILE DEVICE MESSAGES AND ASSOCIATED WITH WEB ADDRESSES;”   (3) U.S. provisional patent application 61/091,330, filed Aug. 22, 2008 and titled “METHODS, APPARATUSES, AND SYSTEMS FOR PROVIDING TIMELY USER CUES PERTAINING TO SPEECH RECOGNITION;”   (4) U.S. nonprovisional patent application Ser. No. 12/198,112, filed Aug. 25, 2008 and titled “FILTERING TRANSCRIPTIONS OF UTTERANCES;”   (5) U.S. nonprovisional patent application Ser. No. 12/198,116, filed Aug. 25, 2008 and titled “FACILITATING PRESENTATION BY MOBILE DEVICE OF ADDITIONAL CONTENT FOR A WORD OR PHRASE UPON UTTERANCE THEREOF;”   (6) U.S. nonprovisional patent application Ser. No. 12/212,644, filed Sep. 17, 2008 and titled “METHODS AND SYSTEMS FOR DYNAMICALLY UPDATING WEB SERVICE PROFILE INFORMATION BY PARSING TRANSCRIBED MESSAGE STRINGS;”   (7) U.S. nonprovisional patent application Ser. No. 12/212,645, filed Sep. 17, 2008 and titled “FACILITATING PRESENTATION OF ADS RELATING TO WORDS OF A MESSAGE;” and   (8) U.S. nonprovisional patent application Ser. No. 12/344,313, filed Dec. 26, 2008 and titled “VALIDATION OF MOBILE ADVERTISING FROM DERIVED INFORMATION.”   (9) U.S. nonprovisional patent application Ser. No. 12/355,319, filed Jan. 16, 2009 and titled “USING A PHYSICAL PHENOMENON DETECTOR TO CONTROL OPERATION OF A SPEECH RECOGNITION ENGINE.”   (10) U.S. nonprovisional patent application Ser. No. 12/400,723, filed Mar. 9, 2009 and titled “USE OF INTERMEDIATE SPEECH TRANSCRIPTION RESULTS IN EDITING FINAL SPEECH TRANSCRIPTION RESULTS.”       
 
         [0013]    Finally, the disclosure of provisional applications 60/789,837 and 61/091,330 are contained in Appendix A and Appendix B, respectively, attached hereto and, likewise, are incorporated herein in their entirety by reference and are intended to provide background and technical information with regard to the systems and environments of the inventions of the current provisional patent application. Similarly, the disclosure of the brochure of Appendix C is incorporated herein in its entirety by reference. 
     
    
     COPYRIGHT STATEMENT 
       [0014]    All of the material in this patent document is subject to copyright protection under the copyright laws of the United States and of other countries. The copyright owner has no objection to the facsimile reproduction by anyone of the patent document or the patent disclosure, as it appears in the governmental files or records, but otherwise reserves all copyright rights whatsoever. 
       COMPUTER PROGRAM LISTING 
       [0015]    Submitted concurrently herewith via the USPTO&#39;s electronic filing system, and incorporated herein by reference, is a computer program listing illustrating instructions, routines, and/or other contents of a computer program. The computer program listing is for 2 computer file(s) that represent a portion of an embodiment of the invention. The first file is titled “CallerNameCode.txt”, has a file size of 4,509 bytes, a size on disk of 8,192 bytes and was created on Mar. 31, 2008 at 7:46:34 and last modified at the same time. The second file is titled “CallerNumberCode.txt”, has a file size of 3,306 bytes, a size on disk of 4,096 bytes and was created on Mar. 31, 2008 at 7:46:22 and last modified at the same time. The computer program listing includes source code written in Java. The target hardware for this implementation is any hardware profile that can support a Java platform, or an equivalent platform. 
       BACKGROUND OF THE PRESENT INVENTION 
       [0016]    Automatic Speech Recognition (“ASR”) systems convert spoken audio into text. Recognition accuracy for a particular utterance can vary based on many factors including the audio fidelity of the recorded speech, correctness of the speaker&#39;s pronunciation, and the like. These factors contribute to continuously varying levels of recognition accuracy which can result in several possible transcriptions for a particular utterance. 
         [0017]    Proper nouns are one of the greatest challenges in the field of speech recognition. The sheer number of different names, places, brand names, etc. within one language/culture/country creates a monumental task for speech recognition engines to correctly convert these spoken proper nouns to text. This is compounded when you consider that when users interact with speech recognition engines in their native tough, they can also speak foreign names, places and brands that the speech engines must try to transcribe. An additional level of complexity arises when you also consider the fact that new proper nouns are also constantly being created as humans create new names, new brands are invented, new places to live are developed, etc. 
         [0018]    One theoretical solution that could remedy this situation, thereby enabling ASR engines to convert spoken proper nouns to their textual representation with near perfect accuracy is to create acoustic models and language models which contain all possible proper nouns known to man. Language models (“LMs”), which may include hierarchical language models (“HLMs”), statistical language models (“SLMs”), grammars, and the like, assign probabilities to a sequence of words by means of a probability distribution and try to capture the properties of a language so as to predict the next word in a speech sequence. They are used in conjunction with acoustic models (“AMs”) to convert dictated words to transcribed text. The current state of the art with regard to both creating and updating AMs and LMs requires speech scientists to manually process hundreds to thousands of hours of spoken phrases or words to build AM and LM databases containing phonemes, all of the possible words within a spoken language, and their statistical interrelationships. ASR engines then compare an audio fingerprint against the AMs and LMs with the goal of obtaining a statistically significant match of the spoken audio to its textual representation. There is great expense in this process since a great deal of engineering time is required to generate and update AMs and LMs as languages continue to evolve and new words are continually coined and used in common lexicon. Thus, the work involved in creating AMs and LMs for all proper nouns, not to mention that of constantly updating them, would be a colossal task, and by today&#39;s standards the cost of doing so would far exceed the return on investment. 
         [0019]    Thus, a need exists for simpler techniques for transcribing proper nouns, forming part of an utterance, in an ASR system. Furthermore, once developed, at least some of these techniques may likewise be utilized to more accurately transcribe other portions of utterances. 
       SUMMARY OF THE PRESENT INVENTION 
       [0020]    The following invention describes a solution to the challenges faced in the speech recognition space in terms of recognizing proper nouns when users interact with speech recognition engines. The same invention can be used to more easily convert spoken phone numbers into correctly formatted phone numbers with Arabic numerals after speech engines try to convert this type of audio data. The basic premise of the invention is that it compares personally identifiable data such as a user&#39;s electronic address book and/or Caller/Recipient ID information (in the case of processing voice mail to text) to the n-best results generated by the speech recognition engine for each word that is output by the engine. A goal of this comparison is to correct a possible misrecognition of a spoken proper noun such as a name or company with its proper textual form or a spoken phone number to correctly formatted phone number with Arabic numerals to improve the overall accuracy of the output of the voice recognition system. 
         [0021]    A first aspect of the present invention relates to a method of utilizing metadata stored in a computer-readable medium to assist in the conversion of an audio stream to a text stream. This method includes the steps of converting an audio stream to a text stream using an ASR engine and comparing at least one portion of the text stream obtained from the ASR engine with metadata stored in a computer-readable medium. 
         [0022]    In a feature of this aspect of the invention, the ASR engine utilizes a speech recognition algorithm. In another feature of this aspect of the invention, the speech recognition algorithm comprises a grammar algorithm and/or a transcription algorithm. In another feature of this aspect of the invention, the backend server comprises a text-to-speech engine (TTS) for generating an audio message from a text message. In another feature of this aspect of the invention, the text stream comprises a highest-confidence string, and at least one alternative string. In another feature of this aspect of the invention, the audio stream is a smaller portion of a larger audio stream. In another feature of this aspect of the invention, the audio stream is a portion of a larger audio stream that has been broken up into utterances. 
         [0023]    In another feature of this aspect of the invention, the metadata is stored on a mobile phone. In another feature of this aspect of the invention, the metadata is stored in an address book. In another feature of this aspect of the invention, the metadata is an alphanumeric string. In another feature of this aspect of the invention, the metadata is stored on a personal digital assistant. In another feature of this aspect of the invention, the metadata is stored in a contact list. In another feature of this aspect of the invention, the metadata is stored in a digital or electronic calendar. In another feature of this aspect of the invention, the metadata is stored on a desktop computer. In another feature of this aspect of the invention, the metadata is stored on a laptop computer. 
         [0024]    In another feature of this aspect of the invention, the metadata is stored on a remote server. In another feature of this aspect of the invention, the metadata is a collation of data stored in different locations. In another feature of this aspect of the invention, the metadata is stored on a removable data-storage device. In another feature of this aspect of the invention, the metadata is stored on a tablet computer. In another feature of this aspect of the invention, the metadata is extracted from an incoming phone call. In another feature of this aspect of the invention, the metadata is Caller ID data. In another feature of this aspect of the invention, the metadata is Recipient ID data. 
         [0025]    In another feature of this aspect of the invention, the metadata comprises Arabic numerals. In another feature of this aspect of the invention, the text stream comprises an SMS text message. In another feature of this aspect of the invention, the text stream is formatted in ASCII. In another feature of this aspect of the invention, the audio stream is a binary file. In another feature of this aspect of the invention, the audio stream is a .mp3 file. In another feature of this aspect of the invention, the audio stream is a .wav file. In another feature of this aspect of the invention, the data stream is compressed. In another feature of this aspect of the invention, the data stream is encrypted. 
         [0026]    A second aspect of the present invention relates to a method of converting an audio stream into a text stream, utilizing metadata. In one embodiment, the method includes the steps of initializing a client device so that the client device is capable of communicating with a backend server, recording an audio stream on the client device, transmitting a data stream, comprising the audio stream and metadata, from the client device to a backend server through a client-server communication protocol, converting the transmitted audio stream into a text stream on the backend server, comparing at least one portion of the text string with at least one portion of the metadata, replacing at least one portion of the text stream with at least one portion of the metadata and sending the text stream back to the client device. 
         [0027]    In a feature of this aspect of the invention, the method includes the additional step of forwarding the text stream to one or more recipients. In another feature of this aspect of the invention, the method includes the additional step of displaying the text stream on the client device. In another feature of this aspect of the invention, the method includes the additional step of displaying advertising messages and/or icons on the client device according to keywords contained in the text stream, wherein the keywords are associated with the advertising messages and/or icons. 
         [0028]    In another feature of this aspect of the invention, the method includes the additional step of locating the position of the client device through a global positioning system (GPS). In another feature of this aspect of the invention, the method includes the additional step of listing locations, proximate to the position of the client device, of a target of interest presented in the converted text stream. 
         [0029]    In another feature of this aspect of the invention, the client device includes a microphone, a speaker, and a display. In another feature of this aspect of the invention, the client device is a mobile phone. In another feature of this aspect of the invention, the backend server comprises a plurality of applications. 
         [0030]    In another feature of this aspect of the invention, the client device comprises a keypad having a plurality of buttons, configured such that each button is associated with one of the plurality of applications. In another feature of this aspect of the invention, the client device comprises a user interface (UI) having a plurality of tabs configured so that each tab is associated with a plurality of user preferences. 
         [0031]    In another feature of this aspect of the invention, the step of initializing the client device includes the steps of initializing a desired application from the client device; and logging into a client account in the backend server from the client device. 
         [0032]    In another feature of this aspect of the invention, the backend server comprises an ad filter, SMS filter, obscenity filter, number filter, date filter, and currency filter. In another feature of this aspect of the invention, the client-server communication protocol is HTTP and/or HTTPS. In another feature of this aspect of the invention, the client-server communication is through a communication service provider of the client device and/or the Internet. In another feature of this aspect of the invention, the client-server communication protocol is UDP. In another feature of this aspect of the invention, the client-server communication is through a communications service provider of the client device and/or the Internet. In another feature of this aspect of the invention, the backend server comprises an ASR engine. 
         [0033]    In another feature of this aspect of the invention, the ASR engine utilizes a speech recognition algorithm. In another feature of this aspect of the invention, the speech recognition algorithm comprises a grammar algorithm and/or a transcription algorithm. In another feature of this aspect of the invention, the backend server comprises a text-to-speech engine (TTS) for generating an audio message from a text message. In another feature of this aspect of the invention, the text stream comprises a highest-confidence string, and at least one alternative string. In another feature of this aspect of the invention, the audio stream is a smaller portion of a larger audio stream. In another feature of this aspect of the invention, the audio stream is a portion of a larger audio stream that has been broken up into utterances. 
         [0034]    In another feature of this aspect of the invention, the metadata is stored on a mobile phone. In another feature of this aspect of the invention, the metadata is stored in an address book. In another feature of this aspect of the invention, the metadata is an alphanumeric string. In another feature of this aspect of the invention, the metadata is stored on a personal digital assistant. In another feature of this aspect of the invention, the metadata is stored in a contact list. In another feature of this aspect of the invention, the metadata is stored in a digital or electronic calendar. In another feature of this aspect of the invention, the metadata is stored on a desktop computer. In another feature of this aspect of the invention, the metadata is stored on a laptop computer. 
         [0035]    In another feature of this aspect of the invention, the metadata is stored on a remote server. In another feature of this aspect of the invention, the metadata is a collation of data stored in different locations. In another feature of this aspect of the invention, the metadata is stored on a removable data-storage device. In another feature of this aspect of the invention, the metadata is stored on a tablet computer. In another feature of this aspect of the invention, the metadata is extracted from an incoming phone call. In another feature of this aspect of the invention, the metadata is Caller ID data. In another feature of this aspect of the invention, the metadata is Recipient ID data. 
         [0036]    In another feature of this aspect of the invention, the metadata comprises Arabic numerals. In another feature of this aspect of the invention, the text stream comprises an SMS text message. In another feature of this aspect of the invention, the text stream is formatted in ASCII. In another feature of this aspect of the invention, the audio stream is a binary file. In another feature of this aspect of the invention, the audio stream is a .mp3 file. In another feature of this aspect of the invention, the audio stream is a .wav file. In another feature of this aspect of the invention, the data stream is compressed. In another feature of this aspect of the invention, the data stream is encrypted. 
         [0037]    A second aspect of the present invention relates to a method of converting an audio stream into a text stream, utilizing metadata. This method includes the steps of initializing a client device so that the client device is capable of communicating with a backend server, speaking into a microphone of the client device to create an audio stream, simultaneously transmitting the audio stream from the client device to a backend server through a client-server communication protocol, transmitting metadata from the client device to the backend server through the client-server communication protocol, converting the transmitted audio stream into a text stream on the backend server, comparing at least one portion of the text string with at least one portion of the metadata, replacing at least one portion of the text stream with at least one portion of the metadata and sending the text stream back to the client device. 
         [0038]    In a feature of this aspect of the invention, the method includes the additional step of forwarding the text stream to one or more recipients. In another feature of this aspect of the invention, the method includes the additional step of displaying the text stream on the client device. In another feature of this aspect of the invention, the method includes the additional step of displaying advertising messages and/or icons on the client device according to keywords contained in the text stream, wherein the keywords are associated with the advertising messages and/or icons. 
         [0039]    In another feature of this aspect of the invention, the method includes the additional step of locating the position of the client device through a global positioning system (GPS). In another feature of this aspect of the invention, the method includes the additional step of listing locations, proximate to the position of the client device, of a target of interest presented in the converted text stream. 
         [0040]    In another feature of this aspect of the invention, the client device includes a microphone, a speaker, and a display. In another feature of this aspect of the invention, the client device is a mobile phone. In another feature of this aspect of the invention, the backend server comprises a plurality of applications. 
         [0041]    In another feature of this aspect of the invention, the client device comprises a keypad having a plurality of buttons, configured such that each button is associated with one of the plurality of applications. In another feature of this aspect of the invention, the client device comprises a user interface (UI) having a plurality of tabs configured so that each tab is associated with a plurality of user preferences. 
         [0042]    In another feature of this aspect of the invention, the step of initializing the client device includes the steps of initializing a desired application from the client device; and logging into a client account in the backend server from the client device. 
         [0043]    In another feature of this aspect of the invention, the backend server comprises an ad filter, SMS filter, obscenity filter, number filter, date filter, and currency filter. In another feature of this aspect of the invention, the client-server communication protocol is HTTP and/or HTTPS. In another feature of this aspect of the invention, the client-server communication is through a communication service provider of the client device and/or the Internet. In another feature of this aspect of the invention, the client-server communication protocol is UDP. In another feature of this aspect of the invention, the client-server communication is through a communications service provider of the client device and/or the Internet. In another feature of this aspect of the invention, the backend server comprises an ASR engine. 
         [0044]    In another feature of this aspect of the invention, the ASR engine utilizes a speech recognition algorithm. In another feature of this aspect of the invention, the speech recognition algorithm comprises a grammar algorithm and/or a transcription algorithm. In another feature of this aspect of the invention, the backend server comprises a text-to-speech engine (TTS) for generating an audio message from a text message. In another feature of this aspect of the invention, the text stream comprises a highest-confidence string, and at least one alternative string. In another feature of this aspect of the invention, the audio stream is a smaller portion of a larger audio stream. In another feature of this aspect of the invention, the audio stream is a portion of a larger audio stream that has been broken up into utterances. 
         [0045]    In another feature of this aspect of the invention, the metadata is stored on a mobile phone. In another feature of this aspect of the invention, the metadata is stored in an address book. In another feature of this aspect of the invention, the metadata is an alphanumeric string. In another feature of this aspect of the invention, the metadata is stored on a personal digital assistant. In another feature of this aspect of the invention, the metadata is stored in a contact list. In another feature of this aspect of the invention, the metadata is stored in a digital or electronic calendar. In another feature of this aspect of the invention, the metadata is stored on a desktop computer. In another feature of this aspect of the invention, the metadata is stored on a laptop computer. 
         [0046]    In another feature of this aspect of the invention, the metadata is stored on a remote server. In another feature of this aspect of the invention, the metadata is a collation of data stored in different locations. In another feature of this aspect of the invention, the metadata is stored on a removable data-storage device. In another feature of this aspect of the invention, the metadata is stored on a tablet computer. In another feature of this aspect of the invention, the metadata is extracted from an incoming phone call. In another feature of this aspect of the invention, the metadata is Caller ID data. In another feature of this aspect of the invention, the metadata is Recipient ID data. 
         [0047]    In another feature of this aspect of the invention, the metadata comprises Arabic numerals. In another feature of this aspect of the invention, the text stream comprises an SMS text message. In another feature of this aspect of the invention, the text stream is formatted in ASCII. In another feature of this aspect of the invention, the audio stream is a binary file. In another feature of this aspect of the invention, the audio stream is a .mp3 file. In another feature of this aspect of the invention, the audio stream is a .wav file. In another feature of this aspect of the invention, the data stream is compressed. In another feature of this aspect of the invention, the data stream is encrypted. 
         [0048]    A third aspect of the present invention relates to a method of converting an audio stream into a text stream, utilizing metadata. This method includes the steps of initializing a client device so that the client device is capable of communicating with a backend server, recording an audio stream on the client device, transmitting a data stream, comprising the audio stream from the client device to a backend server through a client-server communication protocol, converting the transmitted audio stream into a text stream on the backend server, sending the text stream back to the client device, comparing, on the client device, at least one portion of the text string with metadata stored on the client device and replacing at least one portion of the text stream with at least one portion of the metadata. 
         [0049]    In a feature of this aspect of the invention, the method includes the additional step of forwarding the text stream to one or more recipients. In another feature of this aspect of the invention, the method includes the additional step of displaying the text stream on the client device. In another feature of this aspect of the invention, the method includes the additional step of displaying advertising messages and/or icons on the client device according to keywords contained in the text stream, wherein the keywords are associated with the advertising messages and/or icons. 
         [0050]    In another feature of this aspect of the invention, the method includes the additional step of locating the position of the client device through a global positioning system (GPS). In another feature of this aspect of the invention, the method includes the additional step of listing locations, proximate to the position of the client device, of a target of interest presented in the converted text stream. 
         [0051]    In another feature of this aspect of the invention, the client device includes a microphone, a speaker, and a display. In another feature of this aspect of the invention, the client device is a mobile phone. In another feature of this aspect of the invention, the backend server comprises a plurality of applications. 
         [0052]    In another feature of this aspect of the invention, the client device comprises a keypad having a plurality of buttons, configured such that each button is associated with one of the plurality of applications. In another feature of this aspect of the invention, the client device comprises a user interface (UI) having a plurality of tabs configured so that each tab is associated with a plurality of user preferences. 
         [0053]    In another feature of this aspect of the invention, the step of initializing the client device includes the steps of initializing a desired application from the client device; and logging into a client account in the backend server from the client device. 
         [0054]    In another feature of this aspect of the invention, the backend server comprises an ad filter, SMS filter, obscenity filter, number filter, date filter, and currency filter. In another feature of this aspect of the invention, the client-server communication protocol is HTTP and/or HTTPS. In another feature of this aspect of the invention, the client-server communication is through a communication service provider of the client device and/or the Internet. In another feature of this aspect of the invention, the client-server communication protocol is UDP. In another feature of this aspect of the invention, the client-server communication is through a communications service provider of the client device and/or the Internet. In another feature of this aspect of the invention, the backend server comprises an ASR engine. 
         [0055]    In another feature of this aspect of the invention, the ASR engine utilizes a speech recognition algorithm. In another feature of this aspect of the invention, the speech recognition algorithm comprises a grammar algorithm and/or a transcription algorithm. In another feature of this aspect of the invention, the backend server comprises a text-to-speech engine (TTS) for generating an audio message from a text message. In another feature of this aspect of the invention, the text stream comprises a highest-confidence string, and at least one alternative string. In another feature of this aspect of the invention, the audio stream is a smaller portion of a larger audio stream. In another feature of this aspect of the invention, the audio stream is a portion of a larger audio stream that has been broken up into utterances. 
         [0056]    In another feature of this aspect of the invention, the metadata is stored on a mobile phone. In another feature of this aspect of the invention, the metadata is stored in an address book. In another feature of this aspect of the invention, the metadata is an alphanumeric string. In another feature of this aspect of the invention, the metadata is stored on a personal digital assistant. In another feature of this aspect of the invention, the metadata is stored in a contact list. In another feature of this aspect of the invention, the metadata is stored in a digital or electronic calendar. In another feature of this aspect of the invention, the metadata is stored on a desktop computer. In another feature of this aspect of the invention, the metadata is stored on a laptop computer. 
         [0057]    In another feature of this aspect of the invention, the metadata is stored on a remote server. In another feature of this aspect of the invention, the metadata is a collation of data stored in different locations. In another feature of this aspect of the invention, the metadata is stored on a removable data-storage device. In another feature of this aspect of the invention, the metadata is stored on a tablet computer. In another feature of this aspect of the invention, the metadata is extracted from an incoming phone call. In another feature of this aspect of the invention, the metadata is Caller ID data. In another feature of this aspect of the invention, the metadata is Recipient ID data. 
         [0058]    In another feature of this aspect of the invention, the metadata comprises Arabic numerals. In another feature of this aspect of the invention, the text stream comprises an SMS text message. In another feature of this aspect of the invention, the text stream is formatted in ASCII. In another feature of this aspect of the invention, the audio stream is a binary file. In another feature of this aspect of the invention, the audio stream is a .mp3 file. In another feature of this aspect of the invention, the audio stream is a .wav file. In another feature of this aspect of the invention, the data stream is compressed. In another feature of this aspect of the invention, the data stream is encrypted. 
         [0059]    A fourth aspect of the present invention relates to a method of converting an audio stream into a text stream, utilizing metadata. This method includes the steps of recording an audio stream on a client device, converting the audio stream into a text stream, comparing at least one portion of the text string with metadata stored on the client device and replacing at least one portion of the text stream with at least one portion of the metadata. 
         [0060]    In a feature of this aspect of the invention, the method includes the additional step of forwarding the text stream to one or more recipients. In another feature of this aspect of the invention, the method includes the additional step of displaying the text stream on the client device. In another feature of this aspect of the invention, the method includes the additional step of displaying advertising messages and/or icons on the client device according to keywords contained in the text stream, wherein the keywords are associated with the advertising messages and/or icons. 
         [0061]    In another feature of this aspect of the invention, the method includes the additional step of locating the position of the client device through a global positioning system (GPS). In another feature of this aspect of the invention, the method includes the additional step of listing locations, proximate to the position of the client device, of a target of interest presented in the converted text stream. 
         [0062]    In another feature of this aspect of the invention, the client device includes a microphone, a speaker, and a display. In another feature of this aspect of the invention, the client device is a mobile phone. In another feature of this aspect of the invention, the backend server comprises a plurality of applications. 
         [0063]    In another feature of this aspect of the invention, the client device comprises a keypad having a plurality of buttons, configured such that each button is associated with one of the plurality of applications. In another feature of this aspect of the invention, the client device comprises a user interface (UI) having a plurality of tabs configured so that each tab is associated with a plurality of user preferences. 
         [0064]    In another feature of this aspect of the invention, the step of initializing the client device includes the steps of initializing a desired application from the client device; and logging into a client account in the backend server from the client device. 
         [0065]    In another feature of this aspect of the invention, the backend server comprises an ad filter, SMS filter, obscenity filter, number filter, date filter, and currency filter. In another feature of this aspect of the invention, the client-server communication protocol is HTTP and/or HTTPS. In another feature of this aspect of the invention, the client-server communication is through a communication service provider of the client device and/or the Internet. In another feature of this aspect of the invention, the client-server communication protocol is UDP. In another feature of this aspect of the invention, the client-server communication is through a communications service provider of the client device and/or the Internet. In another feature of this aspect of the invention, the backend server comprises an ASR engine. 
         [0066]    In another feature of this aspect of the invention, the ASR engine utilizes a speech recognition algorithm. In another feature of this aspect of the invention, the speech recognition algorithm comprises a grammar algorithm and/or a transcription algorithm. In another feature of this aspect of the invention, the backend server comprises a text-to-speech engine (TTS) for generating an audio message from a text message. In another feature of this aspect of the invention, the text stream comprises a highest-confidence string, and at least one alternative string. In another feature of this aspect of the invention, the audio stream is a smaller portion of a larger audio stream. In another feature of this aspect of the invention, the audio stream is a portion of a larger audio stream that has been broken up into utterances. 
         [0067]    In another feature of this aspect of the invention, the metadata is stored on a mobile phone. In another feature of this aspect of the invention, the metadata is stored in an address book. In another feature of this aspect of the invention, the metadata is an alphanumeric string. In another feature of this aspect of the invention, the metadata is stored on a personal digital assistant. In another feature of this aspect of the invention, the metadata is stored in a contact list. In another feature of this aspect of the invention, the metadata is stored in a digital or electronic calendar. In another feature of this aspect of the invention, the metadata is stored on a desktop computer. In another feature of this aspect of the invention, the metadata is stored on a laptop computer. 
         [0068]    In another feature of this aspect of the invention, the metadata is stored on a remote server. In another feature of this aspect of the invention, the metadata is a collation of data stored in different locations. In another feature of this aspect of the invention, the metadata is stored on a removable data-storage device. In another feature of this aspect of the invention, the metadata is stored on a tablet computer. In another feature of this aspect of the invention, the metadata is extracted from an incoming phone call. In another feature of this aspect of the invention, the metadata is Caller ID data. In another feature of this aspect of the invention, the metadata is Recipient ID data. 
         [0069]    In another feature of this aspect of the invention, the metadata comprises Arabic numerals. In another feature of this aspect of the invention, the text stream comprises an SMS text message. In another feature of this aspect of the invention, the text stream is formatted in ASCII. In another feature of this aspect of the invention, the audio stream is a binary file. In another feature of this aspect of the invention, the audio stream is a .mp3 file. In another feature of this aspect of the invention, the audio stream is a .wav file. In another feature of this aspect of the invention, the data stream is compressed. In another feature of this aspect of the invention, the data stream is encrypted. 
         [0070]    A fourth aspect of the present invention relates to a method of converting an audio stream into a text stream, utilizing metadata. This method includes the steps of transmitting a data stream, comprising an audio stream and metadata, from a client device to a backend server through a client-server communication protocol, converting the transmitted audio stream into a text stream on the backend server, comparing at least one portion of the text string with at least one portion of the metadata, replacing at least one portion of the text stream with at least one portion of the metadata and sending the text stream back to the client device. 
         [0071]    In a feature of this aspect of the invention, the method includes the additional steps, to be performed before transmitting the data stream, of initializing the client device so that the client device is capable of communicating with the backend server and recording the audio stream on the client device. 
         [0072]    In a feature of this aspect of the invention, the method includes the additional step of forwarding the text stream to one or more recipients. In another feature of this aspect of the invention, the method includes the additional step of displaying the text stream on the client device. In another feature of this aspect of the invention, the method includes the additional step of displaying advertising messages and/or icons on the client device according to keywords contained in the text stream, wherein the keywords are associated with the advertising messages and/or icons. 
         [0073]    In another feature of this aspect of the invention, the method includes the additional step of locating the position of the client device through a global positioning system (GPS). In another feature of this aspect of the invention, the method includes the additional step of listing locations, proximate to the position of the client device, of a target of interest presented in the converted text stream. 
         [0074]    In another feature of this aspect of the invention, the client device includes a microphone, a speaker, and a display. In another feature of this aspect of the invention, the client device is a mobile phone. In another feature of this aspect of the invention, the backend server comprises a plurality of applications. 
         [0075]    In another feature of this aspect of the invention, the client device comprises a keypad having a plurality of buttons, configured such that each button is associated with one of the plurality of applications. In another feature of this aspect of the invention, the client device comprises a user interface (UI) having a plurality of tabs configured so that each tab is associated with a plurality of user preferences. 
         [0076]    In another feature of this aspect of the invention, the step of initializing the client device includes the steps of initializing a desired application from the client device; and logging into a client account in the backend server from the client device. 
         [0077]    In another feature of this aspect of the invention, the backend server comprises an ad filter, SMS filter, obscenity filter, number filter, date filter, and currency filter. 
         [0078]    In another feature of this aspect of the invention, the client-server communication protocol is HTTP and/or HTTPS. In another feature of this aspect of the invention, the client-server communication is through a communication service provider of the client device and/or the Internet. In another feature of this aspect of the invention, the client-server communication protocol is UDP. In another feature of this aspect of the invention, the client-server communication is through a communications service provider of the client device and/or the Internet. In another feature of this aspect of the invention, the backend server comprises an ASR engine. 
         [0079]    In another feature of this aspect of the invention, the ASR engine utilizes a speech recognition algorithm. In another feature of this aspect of the invention, the speech recognition algorithm comprises a grammar algorithm and/or a transcription algorithm. In another feature of this aspect of the invention, the backend server comprises a text-to-speech engine (TTS) for generating an audio message from a text message. In another feature of this aspect of the invention, the text stream comprises a highest-confidence string, and at least one alternative string. In another feature of this aspect of the invention, the audio stream is a smaller portion of a larger audio stream. In another feature of this aspect of the invention, the audio stream is a portion of a larger audio stream that has been broken up into utterances. 
         [0080]    In another feature of this aspect of the invention, the metadata is stored on a mobile phone. In another feature of this aspect of the invention, the metadata is stored in an address book. In another feature of this aspect of the invention, the metadata is an alphanumeric string. In another feature of this aspect of the invention, the metadata is stored on a personal digital assistant. In another feature of this aspect of the invention, the metadata is stored in a contact list. In another feature of this aspect of the invention, the metadata is stored in a digital or electronic calendar. In another feature of this aspect of the invention, the metadata is stored on a desktop computer. In another feature of this aspect of the invention, the metadata is stored on a laptop computer. 
         [0081]    In another feature of this aspect of the invention, the metadata is stored on a remote server. In another feature of this aspect of the invention, the metadata is a collation of data stored in different locations. In another feature of this aspect of the invention, the metadata is stored on a removable data-storage device. In another feature of this aspect of the invention, the metadata is stored on a tablet computer. In another feature of this aspect of the invention, the metadata is extracted from an incoming phone call. In another feature of this aspect of the invention, the metadata is Caller ID data. In another feature of this aspect of the invention, the metadata is Recipient ID data. 
         [0082]    In another feature of this aspect of the invention, the metadata comprises Arabic numerals. In another feature of this aspect of the invention, the text stream comprises an SMS text message. In another feature of this aspect of the invention, the text stream is formatted in ASCII. In another feature of this aspect of the invention, the audio stream is a binary file. In another feature of this aspect of the invention, the audio stream is a .mp3 file. In another feature of this aspect of the invention, the audio stream is a .wav file. In another feature of this aspect of the invention, the data stream is compressed. In another feature of this aspect of the invention, the data stream is encrypted. 
         [0083]    A seventh aspect of this invention relates to software stored in a computer-readable medium for causing a client device and/or a backend server to perform functions. These functions include initializing a client device so that the client device is capable of communicating with a backend server, recording an audio stream on the client device, transmitting a data stream, comprising the audio stream and metadata, from the client device to a backend server through a client-server communication protocol, converting the transmitted audio stream into a text stream on the backend server, comparing at least one portion of the text string with at least one portion of the metadata, replacing at least one portion of the text stream with at least one portion of the metadata and sending the text stream back to the client device. 
         [0084]    In a feature of this aspect of the invention, the software includes functions for forwarding the text stream to one or more recipients. In another feature of this aspect of the invention, the software includes a function for displaying the text stream on the client device. In another feature of this aspect of the invention, the software includes a function for displaying advertising messages and/or icons on the client device according to keywords contained in the text stream, wherein the keywords are associated with the advertising messages and/or icons. 
         [0085]    In another feature of this aspect of the invention, the software includes a function for locating the position of the client device through a global positioning system (GPS). 
         [0086]    In another feature of this aspect of the invention, the software includes a function for listing locations, proximate to the position of the client device, of a target of interest presented in the converted text stream. 
         [0087]    In another feature of this aspect of the invention, the client device includes a microphone, a speaker, and a display. In another feature of this aspect of the invention, the client device is a mobile phone. In another feature of this aspect of the invention, the backend server comprises a plurality of applications. 
         [0088]    In another feature of this aspect of the invention, the client device comprises a keypad having a plurality of buttons, configured such that each button is associated with one of the plurality of applications. In another feature of this aspect of the invention, the client device comprises a user interface (UT) having a plurality of tabs configured so that each tab is associated with a plurality of user preferences. 
         [0089]    In another feature of this aspect of the invention, the step of initializing the client device includes the steps of initializing a desired application from the client device; and logging into a client account in the backend server from the client device. 
         [0090]    In another feature of this aspect of the invention, the backend server comprises an ad filter, SMS filter, obscenity filter, number filter, date filter, and currency filter. 
         [0091]    In another feature of this aspect of the invention, the client-server communication protocol is HTTP and/or HTTPS. In another feature of this aspect of the invention, the client-server communication is through a communication service provider of the client device and/or the Internet. In another feature of this aspect of the invention, the client-server communication protocol is UDP. In another feature of this aspect of the invention, the client-server communication is through a communications service provider of the client device and/or the Internet. In another feature of this aspect of the invention, the backend server comprises an ASR engine. 
         [0092]    In another feature of this aspect of the invention, the ASR engine utilizes a speech recognition algorithm. In another feature of this aspect of the invention, the speech recognition algorithm comprises a grammar algorithm and/or a transcription algorithm. In another feature of this aspect of the invention, the backend server comprises a text-to-speech engine (TTS) for generating an audio message from a text message. In another feature of this aspect of the invention, the text stream comprises a highest-confidence string, and at least one alternative string. In another feature of this aspect of the invention, the audio stream is a smaller portion of a larger audio stream. In another feature of this aspect of the invention, the audio stream is a portion of a larger audio stream that has been broken up into utterances. 
         [0093]    In another feature of this aspect of the invention, the metadata is stored on a mobile phone. In another feature of this aspect of the invention, the metadata is stored in an address book. In another feature of this aspect of the invention, the metadata is an alphanumeric string. In another feature of this aspect of the invention, the metadata is stored on a personal digital assistant. In another feature of this aspect of the invention, the metadata is stored in a contact list. In another feature of this aspect of the invention, the metadata is stored in a digital or electronic calendar. In another feature of this aspect of the invention, the metadata is stored on a desktop computer. In another feature of this aspect of the invention, the metadata is stored on a laptop computer. 
         [0094]    In another feature of this aspect of the invention, the metadata is stored on a remote server. In another feature of this aspect of the invention, the metadata is a collation of data stored in different locations. In another feature of this aspect of the invention, the metadata is stored on a removable data-storage device. In another feature of this aspect of the invention, the metadata is stored on a tablet computer. In another feature of this aspect of the invention, the metadata is extracted from an incoming phone call. In another feature of this aspect of the invention, the metadata is Caller ID data. In another feature of this aspect of the invention, the metadata is Recipient ID data. 
         [0095]    In another feature of this aspect of the invention, the metadata comprises Arabic numerals. In another feature of this aspect of the invention, the text stream comprises an SMS text message. In another feature of this aspect of the invention, the text stream is formatted in ASCII. In another feature of this aspect of the invention, the audio stream is a binary file. In another feature of this aspect of the invention, the audio stream is a .mp3 file. In another feature of this aspect of the invention, the audio stream is a .wav file. In another feature of this aspect of the invention, the data stream is compressed. In another feature of this aspect of the invention, the data stream is encrypted. 
         [0096]    An eighth aspect of this invention relates to a system for converting an audio stream into a text stream, utilizing metadata. This system includes a client device, a backend server and software in the client device and backend server for causing the client device and/or backend server to perform functions. These functions include initializing a client device so that the client device is capable of communicating with a backend server, recording an audio stream on the client device, transmitting a data stream, comprising the audio stream and metadata, from the client device to a backend server through a client-server communication protocol, converting the transmitted audio stream into a text stream on the backend server, comparing at least one portion of the text string with at least one portion of the metadata, replacing at least one portion of the text stream with at least one portion of the metadata and sending the text stream back to the client device. 
         [0097]    In a feature of this aspect of the invention, the software includes functions for forwarding the text stream to one or more recipients. In another feature of this aspect of the invention, the software includes a function for displaying the text stream on the client device. In another feature of this aspect of the invention, the software includes a function for displaying advertising messages and/or icons on the client device according to keywords contained in the text stream, wherein the keywords are associated with the advertising messages and/or icons. 
         [0098]    In another feature of this aspect of the invention, the software includes a function for locating the position of the client device through a global positioning system (GPS). 
         [0099]    In another feature of this aspect of the invention, the software includes a function for listing locations, proximate to the position of the client device, of a target of interest presented in the converted text stream. 
         [0100]    In another feature of this aspect of the invention, the client device includes a microphone, a speaker, and a display. In another feature of this aspect of the invention, the client device is a mobile phone. In another feature of this aspect of the invention, the backend server comprises a plurality of applications. 
         [0101]    In another feature of this aspect of the invention, the client device comprises a keypad having a plurality of buttons, configured such that each button is associated with one of the plurality of applications. In another feature of this aspect of the invention, the client device comprises a user interface (UI) having a plurality of tabs configured so that each tab is associated with a plurality of user preferences. 
         [0102]    In another feature of this aspect of the invention, the step of initializing the client device includes the steps of initializing a desired application from the client device; and logging into a client account in the backend server from the client device. 
         [0103]    In another feature of this aspect of the invention, the backend server comprises an ad filter, SMS filter, obscenity filter, number filter, date filter, and currency filter. 
         [0104]    In another feature of this aspect of the invention, the client-server communication protocol is HTTP and/or HTTPS. In another feature of this aspect of the invention, the client-server communication is through a communication service provider of the client device and/or the Internet. In another feature of this aspect of the invention, the client-server communication protocol is UDP. In another feature of this aspect of the invention, the client-server communication is through a communications service provider of the client device and/or the Internet. In another feature of this aspect of the invention, the backend server comprises an ASR engine. 
         [0105]    In another feature of this aspect of the invention, the ASR engine utilizes a speech recognition algorithm. In another feature of this aspect of the invention, the speech recognition algorithm comprises a grammar algorithm and/or a transcription algorithm. In another feature of this aspect of the invention, the backend server comprises a text-to-speech engine (TTS) for generating an audio message from a text message. In another feature of this aspect of the invention, the text stream comprises a highest-confidence string, and at least one alternative string. In another feature of this aspect of the invention, the audio stream is a smaller portion of a larger audio stream. In another feature of this aspect of the invention, the audio stream is a portion of a larger audio stream that has been broken up into utterances. 
         [0106]    In another feature of this aspect of the invention, the metadata is stored on a mobile phone. In another feature of this aspect of the invention, the metadata is stored in an address book. In another feature of this aspect of the invention, the metadata is an alphanumeric string. In another feature of this aspect of the invention, the metadata is stored on a personal digital assistant. In another feature of this aspect of the invention, the metadata is stored in a contact list. In another feature of this aspect of the invention, the metadata is stored in a digital or electronic calendar. In another feature of this aspect of the invention, the metadata is stored on a desktop computer. In another feature of this aspect of the invention, the metadata is stored on a laptop computer. 
         [0107]    In another feature of this aspect of the invention, the metadata is stored on a remote server. In another feature of this aspect of the invention, the metadata is a collation of data stored in different locations. In another feature of this aspect of the invention, the metadata is stored on a removable data-storage device. In another feature of this aspect of the invention, the metadata is stored on a tablet computer. In another feature of this aspect of the invention, the metadata is extracted from an incoming phone call. In another feature of this aspect of the invention, the metadata is Caller ID data. In another feature of this aspect of the invention, the metadata is Recipient ID data. 
         [0108]    In another feature of this aspect of the invention, the metadata comprises Arabic numerals. In another feature of this aspect of the invention, the text stream comprises an SMS text message. In another feature of this aspect of the invention, the text stream is formatted in ASCII. In another feature of this aspect of the invention, the audio stream is a binary file. In another feature of this aspect of the invention, the audio stream is a .mp3 file. In another feature of this aspect of the invention, the audio stream is a .wav file. In another feature of this aspect of the invention, the data stream is compressed. In another feature of this aspect of the invention, the data stream is encrypted. 
         [0109]    A further aspect of the present invention relates to a method of utilizing metadata stored in a computer-readable medium to assist in the conversion of spoken audio input, received by a hand-held mobile communication device, into a textual representation for display on the hand-held mobile communication device. This method includes the steps of initializing a hand-held mobile communication device so that the hand-held mobile communication device is capable of communicating with a backend server via a data channel of the hand-held mobile communication device, upon receipt of an utterance by the hand-held mobile communication device, recording and storing an audio message, representative of the utterance, in the hand-held mobile communication device in the form of binary audio data, transmitting, via the data channel, the recorded and stored binary audio data, representing the utterance, from the hand-held mobile communication device to a backend server through a client-server communication protocol, in conjunction with the transmission of the recorded and stored binary audio data, transmitting metadata from the hand-held mobile communication device to the backend server through the client-server communication protocol, converting the transmitted binary audio data into a textual representation of the utterance in the backend server, comparing at least one portion of the textual representation at least one portion of the metadata, replacing at least one portion of the textual representation with at least one portion of the metadata, and sending the converted textual representation of the utterance, with metadata replacement, from the server back to the hand-held mobile communication device. 
         [0110]    In a feature of this aspect of the invention, the method includes the additional step of processing the converted textual representation of the utterance with a Digit Filter that substitutes Arabic numerals for words representing particular numbers or that are homophones of words representing particular numbers. In another feature, the step of processing the converted textual representation of the utterance with a Digit Filter occurs before the comparing and replacing steps. Alternatively, in another feature the step of processing the converted textual representation of the utterance with a Digit Filter occurs after the comparing and replacing steps. 
         [0111]    In yet another feature of this aspect of the present invention the method further includes a step of processing the converted textual representation of the utterance with a Telephone Filter that formats the Arabic numerals of a telephone number into a conventional format. In a further feature, the step of processing the converted textual representation of the utterance with a Telephone Filter occurs before the comparing and replacing steps. Alternatively, in another feature, the step of processing the converted textual representation of the utterance with a Telephone Filter occurs after the comparing and replacing steps. 
         [0112]    A feature of this aspect of the present inventions includes a step of forwarding the converted textual representation, with metadata replacement, from the hand-held mobile communication device to one or more recipients. Another feature of this aspect includes a step of displaying the converted textual representation, with metadata replacement, on the hand-held mobile communication device. In yet another feature the client device comprises a mobile phone. In another feature the client-server communication protocol is HTTP and/or HTTPS. In another feature the client-server communication protocol is UDP. 
         [0113]    In another feature of this aspect of the present invention, the backend server includes an automatic speech recognition (ASR) engine. In a further feature, the ASR engine utilizes a speech recognition algorithm. In yet another feature the speech recognition algorithm comprises a grammar algorithm and/or a transcription algorithm. In another feature, the backend server includes a text-to-speech engine (TTS) for generating an audio message from a text message. 
         [0114]    In a feature of this aspect of the invention, the text stream includes a highest-confidence string, and at least one alternative string. In another feature, the metadata is stored on a mobile phone. In yet another feature the metadata is stored in an address book. In a further feature, the metadata is in an alphanumeric string. In yet another feature the metadata is stored in a contact list. In another feature the metadata is stored in a digital or electronic calendar. In still another feature, the metadata is a collation of data stored in different locations. In yet still another feature the metadata is extracted from an incoming phone call. In another feature the metadata is Caller ID data. In still another feature, the metadata is Recipient ID data. In a further feature, the metadata comprises Arabic numerals. In another feature, the binary audio data is a binary file. In still another feature, the binary audio data is a .mp3 file. In another feature, the binary audio data is a .wav file. 
         [0115]    In a feature of this aspect of the present invention, the binary audio data and the metadata form a single data stream. In another feature, the data stream is compressed. In yet another feature, the data stream is encrypted. 
         [0116]    Another feature of this aspect of the present invention includes a step of displaying advertising messages and/or icons on the hand-held mobile communication device according to keywords contained in the textual representation of the utterance, wherein the keywords are associated with the advertising messages and/or icons. A further feature includes the additional step of locating a geospatial position of the hand-held mobile communication device through a global positioning system (GPS). Still another feature includes the additional step of listing locations, proximate to the position of the client device, of a target of interest presented in the converted text stream. 
         [0117]    In another feature of this aspect of the invention, the backend server includes a plurality of applications. In another feature, the client device includes a keypad having a plurality of buttons, configured such that each button is associated with one of the plurality of applications. In an alternative feature the client device includes a user interface (UI) having a plurality of tabs configured so that each tab is associated with a plurality of user preferences. In yet another alternative feature, the step of initializing the client device includes the steps of initializing a desired application from the client device, and logging into a client account in the backend server from the client device. In yet another alternative feature, the backend server includes an ad filter, an SMS filter, an obscenity filter, a number filter, a data filter, and a currency filter. 
         [0118]    Another aspect of the present invention relates to a method of utilizing metadata stored in a computer-readable medium to assist in the conversion of spoken audio input, received by a hand-held mobile communication device, into a textual representation for display on the hand-held mobile communication device. The method includes the steps of initializing a hand-held mobile communication device so that the hand-held mobile communication device is capable of communicating with a backend server via a data channel of the hand-held mobile communication device, upon receipt of an utterance by the hand-held mobile communication device, recording and storing an audio message, representative of the utterance, in the hand-held mobile communication device in the form of binary audio data, transmitting, via the data channel, the recorded and stored binary audio data, representing the utterance, from the hand-held mobile communication device to a backend server through a client-server communication protocol, storing metadata, associated with the hand-held mobile communication device, at the backend server, converting the transmitted binary audio data into a textual representation of the utterance in the backend server, comparing at least one portion of the textual representation at least one portion of the metadata, replacing at least one portion of the textual representation with at least one portion of the metadata, and sending the converted textual representation of the utterance, with metadata replacement, from the server back to the hand-held mobile communication device. 
         [0119]    A feature of this aspect of the invention further includes a step of forwarding the converted textual representation, with metadata replacement, from the hand-held mobile communication device to one or more recipients. 
         [0120]    Another aspect of the present invention relates to a system for utilizing metadata stored in a computer-readable medium to assist in the conversion of spoken audio input, received by a hand-held mobile communication device, into a textual representation for display on the hand-held mobile communication device. The system includes a hand-held mobile communication device, a backend server, and software in the hand-held mobile communication device and backend server for causing the hand-held mobile communication device and/or the backend server to perform functions. Such functions to be performed include initializing the hand-held mobile communication device so that the hand-held mobile communication device is capable of communicating with the backend server via a data channel of the hand-held mobile communication device, upon receipt of an utterance by the hand-held mobile communication device, recording and storing an audio message, representative of the utterance, in the hand-held mobile communication device in the form of binary audio data, and transmitting, via the data channel, the recorded and stored binary audio data, representing the utterance, from the hand-held mobile communication device to a backend server through a client-server communication protocol. Further functions include, in conjunction with the transmission of the recorded and stored binary audio data, transmitting metadata from the hand-held mobile communication device to the backend server through the client-server communication protocol, converting the transmitted binary audio data into a textual representation of the utterance in the backend server, comparing at least one portion of the textual representation at least one portion of the metadata, replacing at least one portion of the textual representation with at least one portion of the metadata, and sending the converted textual representation of the utterance, with metadata replacement, from the server back to the hand-held mobile communication device. 
     
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
         [0121]    Further aspects, features, embodiments, and advantages of the present invention will become apparent from the following detailed description with reference to the drawings, wherein: 
           [0122]      FIG. 1  is a block diagram of a communication system in accordance with a preferred embodiment of the present invention; 
           [0123]      FIG. 2  is a block diagram of a communication system in accordance with another preferred embodiment of the present invention; 
           [0124]      FIG. 3  is a block diagram illustrating communications between two users via a portion of the communication system of  FIG. 1 ; 
           [0125]      FIG. 4  is a block diagram of an exemplary implementation of the system of  FIG. 1 ; 
           [0126]      FIG. 5  is a table listing various possible transcription results for each fragment or portion of the utterance; 
           [0127]      FIGS. 6A-6E  are graphical depictions, on a transmitting device, of the transcription of the utterance of  FIG. 3 , illustrating the selection and replacement of one of the fragment results reached by the ASR engine for one portion of the utterance; 
           [0128]      FIG. 7  is a graphical depiction, on a transmitting device, of the transcription of the utterance of  FIG. 3 , illustrating the various alternatives reached by the ASR engine for each portion of the utterance; 
           [0129]      FIG. 8A  is a high-level flowchart illustrating the operation of such a process; 
           [0130]      FIG. 8B  is a high-level flowchart illustrating the operation of another such process; 
           [0131]      FIG. 8C  is a high-level flowchart illustrating the operation of another such process; 
           [0132]      FIGS. 9A and 9B  are tables listing various possible transcription results for each of several fragments or portions of the exemplary utterance; 
           [0133]      FIG. 10  is a block diagram of the system architecture of one commercial implementation; 
           [0134]      FIG. 11  is a block diagram of a portion of  FIG. 10 ; 
           [0135]      FIG. 12  is a typical header section of an HTTP request from the client in the commercial implementation; 
           [0136]      FIG. 13  illustrates exemplary protocol details for a request for a location of a login server and a subsequent response; 
           [0137]      FIG. 14  illustrates exemplary protocol details for a login request and a subsequent response; 
           [0138]      FIG. 15  illustrates exemplary protocol details for a submit request and a subsequent response; 
           [0139]      FIG. 16  illustrates exemplary protocol details for a results request and a subsequent response; 
           [0140]      FIG. 17  illustrates exemplary protocol details for an XML hierarchy returned in response to a results request; 
           [0141]      FIG. 18  illustrates exemplary protocol details for a text to speech request and a subsequent response; 
           [0142]      FIG. 19  illustrates exemplary protocol details for a correct request; 
           [0143]      FIG. 20  illustrates exemplary protocol details for a ping request; and 
           [0144]      FIG. 21  illustrates exemplary protocol details for a debug request. 
       
    
    
     DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS 
       [0145]    As a preliminary matter, it will readily be understood by one having ordinary skill in the relevant art (“Ordinary Artisan”) that the present invention has broad utility and application. Furthermore, any embodiment discussed and identified as being “preferred” is considered to be part of a best mode contemplated for carrying out the present invention. Other embodiments also may be discussed for additional illustrative purposes in providing a full and enabling disclosure of the present invention. Moreover, many embodiments, such as adaptations, variations, modifications, and equivalent arrangements, will be implicitly disclosed by the embodiments described herein and fall within the scope of the present invention. 
         [0146]    Accordingly, while the present invention is described herein in detail in relation to one or more embodiments, it is to be understood that this disclosure is illustrative and exemplary of the present invention, and is made merely for the purposes of providing a full and enabling disclosure of the present invention. The detailed disclosure herein of one or more embodiments is not intended, nor is it to be construed, to limit the scope of patent protection afforded the present invention, which scope is to be defined by the claims and the equivalents thereof. It is not intended that the scope of patent protection afforded the present invention be defined by reading into any claim a limitation found herein that does not explicitly appear in the claim itself. 
         [0147]    Thus, for example, any sequence(s) and/or temporal order of steps of various processes or methods that are described herein are illustrative and not restrictive. Accordingly, it should be understood that, although steps of various processes or methods may be shown and described as being in a sequence or temporal order, the steps of any such processes or methods are not limited to being carried out in any particular sequence or order, absent an indication otherwise. Indeed, the steps in such processes or methods generally may be carried out in various different sequences and orders while still falling within the scope of the present invention. Accordingly, it is intended that the scope of patent protection afforded the present invention is to be defined by the appended claims rather than the description set forth herein. 
         [0148]    Additionally, it is important to note that each term used herein refers to that which the Ordinary Artisan would understand such term to mean based on the contextual use of such term herein. To the extent that the meaning of a term used herein—as understood by the Ordinary Artisan based on the contextual use of such term—differs in any way from any particular dictionary definition of such term, it is intended that the meaning of the term as understood by the Ordinary Artisan should prevail. 
         [0149]    Furthermore, it is important to note that, as used herein, “a” and “an” each generally denotes “at least one,” but does not exclude a plurality unless the contextual use dictates otherwise. Thus, reference to “a picnic basket having an apple” describes “a picnic basket having at least one apple” as well as “a picnic basket having apples.” In contrast, reference to “a picnic basket having a single apple” describes “a picnic basket having only one apple.” 
         [0150]    When used herein to join a list of items, “or” denotes “at least one of the items,” but does not exclude a plurality of items of the list. Thus, reference to “a picnic basket having cheese or crackers” describes “a picnic basket having cheese without crackers”, “a picnic basket having crackers without cheese”, and “a picnic basket having both cheese and crackers.”Finally, when used herein to join a list of items, “and” denotes “all of the items of the list.”Thus, reference to “a picnic basket having cheese and crackers” describes “a picnic basket having cheese, wherein the picnic basket further has crackers,” as well as describes “a picnic basket having crackers, wherein the picnic basket further has cheese.” 
         [0151]    Referring now to the drawings, in which like numerals represent like components throughout the several views, the preferred embodiments of the present invention are next described. The following description of the preferred embodiment(s) is merely exemplary in nature and is in no way intended to limit the invention, its application, or uses. 
         [0152]      FIG. 1  is a block diagram of a communication system  10  in accordance with a preferred embodiment of the present invention. As shown therein, the communication system  10  includes at least one transmitting device  12  and at least one receiving device  14 , one or more network systems  16  for connecting the transmitting device  12  to the receiving device  14 , and an automatic speech recognition (“ASR”) system  18 , including an ASR engine. Transmitting and receiving devices  12 , 14  may include cell phones  21 , smart phones  22 , PDAs  23 , tablet notebooks  24 , various desktop and laptop computers  25 , 26 , 27 , and the like, one or more of which may be a handheld device. At least some of the handheld devices are capable of communications using both voice and data channels, either separately or simultaneously, and in at least some embodiments, in parallel. One or more of the devices  12 , 14 , such as the illustrated iMac and laptop computers  25 , 26 , may connect to the network systems  16  via a wireless access point  28 . The various devices  12 , 14  (one or both types of which being sometimes referred to herein as “client devices”) may be of any conventional design and manufacture. 
         [0153]      FIG. 2  is a block diagram of a communication system  60  in accordance with another preferred embodiment of the present invention. This system  60  is similar to the system  10  of  FIG. 1 , except that the ASR system  18  of  FIG. 1  has been omitted and the ASR engine has instead been incorporated into the various transmitting devices  12 , including cell phones  61 , smart phones  62 , PDAs  63 , tablet notebooks  64 , various desktop and laptop computers  65 , 66 , 67 , and the like. 
         [0154]    It will be appreciated that the illustrations of  FIGS. 1 and 2  are intended primarily to provide context in which the inventive features of the present invention may be placed. A more complete explanation of one or more system architectures implementing such systems is provided elsewhere herein, in the incorporated applications and/or in the incorporated Appendices attached hereto. Furthermore, in the context of text messaging, the communication systems  10 , 60  each preferably include, inter alia, a telecommunications network. In the context of instant messaging, the communications systems  10 , 60  each preferably include, inter alia, the Internet. 
         [0155]      FIG. 3  is a block diagram illustrating communications between two users  32 , 34  via a portion of the communication system  10  of  FIG. 1 . As shown therein, a first user  32 , sometimes referred to herein as a “transmitting user,” is communicating with a second user  34 , sometimes referred to herein as a “receiving user,” by way of respective transmitting and receiving devices  12 , 14 . More particularly, the transmitting user  32  uses his transmitting device  12  to initiate text messages that are transmitted to, and received by, the receiving user  34  via her receiving device  14 . In the context of text messaging, the transmitting user  32  may send text messages using his transmitting device  12 , for example via SMS, and the receiving user  34  receives text messages on his receiving device  14 , in this case also via SMS. In the context of instant messaging, the transmitting user  32  may send instant messages via an IM client using his transmitting device  12 , and particularly using a data channel of the transmitting device  12 , and the receiving user  34  receives instant messages on her receiving device  14 , via an IM client, particularly using a data channel of the receiving device  14 . In either case, the transmitting user  32  preferably speaks into his transmitting device  12  with his utterances being converted to text for communicating to the receiving device  14 , all as more fully described hereinbelow. 
         [0156]    In either case, the first user  32  speaks an utterance  36  into the transmitting device  12 , and the recorded speech audio is sent to the ASR system  18 , preferably using a data channel of the device  12 . In  FIG. 3 , the utterance  36  is “That man in here is a serial killer.” The ASR engine in the ASR system  18  attempts to recognize and transcribe the speech into text. 
         [0157]    In at least some embodiments, the transmitting user  32  in  FIG. 3  may generate text messages by speaking into his transmitting device  12  and causing his utterances to be converted to text for communicating to the receiving device  14 . One or more systems and methods for carrying out such a process are described, for example, in the aforementioned U.S. Patent Application Pub. No. US 2007/0239837, but are at least partially described herein. More particularly,  FIG. 4  may be understood to be a block diagram of an exemplary implementation of the system  10  of  FIG. 1 . In this implementation, the transmitting device  12  is a mobile phone, the ASR system  18  is implemented in one or more backend servers  160 , and the one or more network systems  16  include transceiver towers  130 , one or more mobile communication service providers  140  (operating under joint or independent control) and the Internet  150 . The backend server  160  is or may be placed in communication with the mobile phone  12  via the mobile communication service provider  140  and the Internet  150 . The mobile phone has a microphone, a speaker and a display. 
         [0158]    A first transceiver tower  130 A is positioned between the mobile phone  12  (or the user  32  of the mobile phone  12 ) and the mobile communication service provider  140 , for receiving an audio message (V 1 ), a text message (T 3 ) and/or a verified text message (V/T 1 ) from one of the mobile phone  12  and the mobile communication service provider  140  and transmitting it (V 2 , T 4 , V/T 2 ) to the other of the mobile phone  12  and the mobile communication service provider  140 . Such communications (V 1 , T 4 , V/T 1 ) to and from the mobile phone  12  are preferably carried out using a data channel, rather than a voice channel, of the mobile phone  12 . A second transceiver tower  130 B is positioned between the mobile communication service provider  140  and mobile devices  170 , generally defined as receiving devices  14  equipped to communicate wirelessly via mobile communication service provider  140 , for receiving a verified text message (V/T 3 ) from the mobile communication service provider  140  and transmitting it (V 5  and T 5 ) to the mobile devices  170 . In at least some embodiments, the mobile devices  170  are adapted for receiving a text message converted from an audio message created in the mobile phone  12 . Additionally, in at least some embodiments, the mobile devices  170  are also capable of receiving an audio message from the mobile phone  12 . The mobile devices  170  include, but are not limited to, a pager, a palm PC, a mobile phone, or the like. 
         [0159]    The system  10  also includes software, as disclosed below in more detail, installed in the mobile phone  12  and the backend server  160  for causing the mobile phone  12  and/or the backend server  160  to perform the following functions. The first step is to initialize the mobile phone  12  to establish communication between the mobile phone  12  and the backend server  160 , which includes initializing a desired application from the mobile phone  12  and logging into a user account in the backend server  160  from the mobile phone  12 . Then, the user  32  presses and holds one of the buttons of the mobile phone  12  and speaks an utterance  36 , thus generating an audio message, V 1 . At this stage, the audio message V 1  is recorded in the mobile phone  12 . By releasing the button, the recorded audio message V 1  is sent to the backend server  160  through the mobile communication service provider  140 . 
         [0160]    In the exemplary embodiment of the present invention as shown in  FIG. 4 , the recorded audio message V 1  is first transmitted to the first transceiver tower  130 A from the mobile phone  12 . The first transceiver tower  130 A outputs the audio message V 1  into an audio message V 2  that is, in turn, transmitted to the mobile communication service provider  140 . Then the mobile communication service provider  140  outputs the audio message V 2  into an audio message V 3  and transmits it (V 3 ) to the Internet  150 . The Internet  150  outputs the audio message V 3  into an audio message V 4  and transmits it (V 4 ) to the backend server  160 . The content of all the audio messages V 1 -V 4  is identical. 
         [0161]    The backend server  160  then converts the audio message V 4  into a text message, T 1 , and/or a digital signal, D 1 , in the backend server  160  by means of a speech recognition algorithm including a grammar algorithm and/or a transcription algorithm. The text message T 1  and the digital signal D 1  correspond to two different formats of the audio message V 4 . The text message T 1  and/or the digital signal D 1  are sent back to the Internet  150  that outputs them into a text message T 2  and a digital signal D 2 , respectively. 
         [0162]    The digital signal D 2  is transmitted to a digital receiver  180 , generally defined as a receiving device  14  equipped to communicate with the Internet and capable of receiving the digital signal D 2 . In at least some embodiments, the digital receiver  180  is adapted for receiving a digital signal converted from an audio message created in the mobile phone  12 . Additionally, in at least some embodiments, the digital receiver  180  is also capable of receiving an audio message from the mobile phone  12 . A conventional computer is one example of a digital receiver  180 . In this context, a digital signal D 2  may represent, for example, an email or instant message. 
         [0163]    It should be understood that, depending upon the configuration of the backend server  160  and software installed on the mobile phone  12 , and potentially based upon the system set up or preferences of the user  32 , the digital signal D 2  can either be transmitted directly from the backend server  160  or it can be provided back to the mobile phone  12  for review and acceptance by the user  32  before it is sent on to the digital receiver  180 . 
         [0164]    The text message T 2  is sent to the mobile communication service provider  140  that outputs it (T 2 ) into a text message T 3 . The output text message T 3  is then transmitted to the first transceiver tower  130 A. The first transceiver tower  130 A then transmits it (T 3 ) to the mobile phone  12  in the form of a text message T 4 . It is noted that the substantive content of all the text messages T 1 -T 4  may be identical, which are the corresponding text form of the audio messages V 1 -V 4 . 
         [0165]    Upon receiving the text message T 4 , the user  32  verifies it and sends the verified text message V/T 1  to the first transceiver tower  130 A that in turn, transmits it to the mobile communication service provider  140  in the form of a verified text V/T 2 . The verified text V/T 2  is transmitted to the second transceiver tower  130 B in the form of a verified text V/T 3  from the mobile communication service provider  140 . Then, the transceiver tower  130 B transmits the verified text V/T 3  to the mobile devices  170 . 
         [0166]    In at least one implementation, the audio message is simultaneously transmitted to the backend server  160  from the mobile phone  12 , when the user  32  speaks to the mobile phone  12 . In this circumstance, it is preferred that no audio message is recorded in the mobile phone  12 , although it is possible that an audio message could be both transmitted and recorded. 
         [0167]    Such a system  10  may be utilized to convert an audio message into a text message. In at least one implementation, this may be accomplished by first initializing a transmitting device so that the transmitting device is capable of communicating with a backend server  160 . Second, a user  32  speaks to or into the client device  12  so as to create a stream of an audio message. The audio message can be recorded and then transmitted to the backend server  160 , or the audio message can be simultaneously transmitted to the backend server  160  through a client-server communication protocol. Streaming may be accomplished according to processes described elsewhere herein and, in particular, in  FIG. 4 , and accompanying text, of the aforementioned U.S. Patent Application Pub. No. US 2007/0239837. The transmitted audio message is converted into the text message in the backend server  160 . The converted text message is then sent back to the client device  12 . Upon the user&#39;s verification, the converted text message is forwarded to one or more recipients  34  and their respective receiving devices  14 , where the converted text message may be displayed on the device  14 . Incoming messages may be handled, for example, according to processes described elsewhere herein and, in particular, in  FIG. 2 , and accompanying text, of the aforementioned U.S. Patent Application Pub. No. US 2007/0239837. 
         [0168]    Still further, in at least one implementation, one or both types of client device  12 , 14  may be located through a global positioning system (GPS); and listing locations, proximate to the position of the client device  12 , 14 , of a target of interest may be presented in the converted text message. 
         [0169]    Furthermore, in converting speech to text, speech transcription performance indications may be provided to the receiving user  34  in accordance with the disclosure of U.S. patent application Ser. No. 12/197,213, filed Aug. 22, 2008 and entitled “CONTINUOUS SPEECH TRANSCRIPTION PERFORMANCE INDICATION,” which, together with any corresponding patent application publications thereof, is hereby incorporated herein by reference. 
         [0170]    Additionally, in the context of SMS messaging, the ASR engine preferably makes use of both statistical language models (SLMs) for returning results from the audio data, and finite grammars used to post-process the text results, in accordance with the disclosure of U.S. patent application Ser. No. 12/198,112, filed Aug. 25, 2008 and entitled “FILTERING TRANSCRIPTIONS OF UTTERANCES,” which, together with any corresponding patent application publications thereof, is incorporated herein by reference. This is believed to result in text messages that are formatted in a way that looks more typical of how a human would have typed the text message using a mobile device. 
         [0171]    The ASR system  18  transmits the text back to the originating device  12 . Once this device  12  receives the transcribed text, it preferably displays the message to the transmitting user  32  for verification and editing as necessary, and upon approval by the user, the text may be formatted into a text message or instant message that may be transmitted to a destination communication device such as the receiving device  14  described previously. If not already selected or pre-set, the user  32  selects a desired phone number or other destination address stored in memory on the transmitting device  12  or in a remote server or inputs a number or other address via one or more input elements, and an outgoing message signal corresponding to the number or address is transmitted. Voice commands may be utilized to control such functionality, and such a process may be implemented, for example, in accordance with the disclosure of U.S. patent application Ser. No. 12/355,319, filed Jan. 16, 2009 and entitled “USING A PHYSICAL PHENOMENON DETECTOR TO CONTROL OPERATION OF A SPEECH RECOGNITION ENGINE,” which, together with any corresponding patent application publications thereof, is incorporated herein by reference. 
         [0172]      FIG. 5  is a table  400  listing various possible transcription results for each fragment or portion of the utterance  36 . More particularly, “that man,” “batman” and “hat man” are listed as possible transcription results for the utterance fragment “That man;” “engineer,” “engine ear,” “in here” and “ear” are presented as possible transcription results for the utterance fragment “in here;” “is a,” “was a” and “was uh” are listed as possible transcription results for the utterance fragment “is a;” “cereal,” “serial,” “see real” and “surreal” are listed as a possible transcription results for the utterance fragment “serial;” and “filler,” “killer,” “biller” and “fuller” are listed as possible transcription results for the utterance fragment “killer.” It will be appreciated that each fragment is presented in conjunction with a confidence level, expressed as a percentage, representing the calculated relative confidence the ASR system  18  has that that fragment is accurate. In  FIG. 5 , the possible transcription results for each fragment or portion of the utterance  36  are presented in descending order of confidence. It will be further appreciated that only those results in which the confidence level is 10% or above are shown in the table  400 , but that many other possible results may be considered as well. 
         [0173]    When transcription is complete, the fragment results having the highest confidence levels are presented to the user  32  in the order in which the utterance fragments were spoken. In particular, for each fragment of the utterance, the fragment result having the highest confidence level is preferably presented to the user  32  such that the overall transcribed utterance represents the most likely transcription result as measured by the confidence levels of the respective fragment results. 
         [0174]    If the transcribed utterance thus presented is not the one desired by the user  32 , the user  32  may then take action to edit or correct the result. For example, the user  32  may choose to edit one or more of the fragment results.  FIGS. 6A-6E  are graphical depictions, on a transmitting device  12 , of the transcription of the utterance  36  of  FIG. 3 , illustrating the selection and replacement of one of the fragment results reached by the ASR engine for one portion of the utterance  36 . In  FIGS. 6A-6E , all of the fragment results having a confidence level of 10% or more are available to the user  32  for selection and replacement of the corresponding portion of the transcribed utterance. In  FIG. 6A , the complete transcribed utterance in which the ASR engine has the greatest confidence is first presented to the user  32 . It will be appreciated that this proposed transcription is not accurate, in that the ASR engine has returned “engineer” as the most likely fragment result for “in here,” “cereal” as the most likely fragment result for “serial,” and “filler” as the most likely fragment result for “killer.” 
         [0175]    In  FIG. 6B , the user  32  begins correcting the transcribed utterance by selecting one of the fragment results for editing or correction. In particular, the user  32  has selected the utterance fragment with the final result “engineer” for replacement, thereby causing a drop-down menu or list to appear. Referring again to  FIG. 5 , it is to be understood that the items in the drop-down list in  FIG. 6B  are preferably the fragment results for that portion of the utterance  36  that had confidence levels of 10% or more. In  FIG. 6C , the user  32  has selected one of those alternative fragment results (“in here”) to replace the one returned by the ASR engine. In  FIG. 6D , the user  32  has gone ahead and replaced “engineer” with “in here,” producing the edited final result of “that man in here is a cereal filler.” By further using the same process to replace the final result “cereal” with “serial” and the final result “filler” with “killer,” the user  32  is able to reach the intended result of “that man in here is a serial killer,” as shown in  FIG. 6E . Selection may be accomplished in any conventional manner, including manipulation of input keys, buttons or the like on the device  12 , use of a stylus or other external tool, voice activation, or the like. 
         [0176]    In another approach, the user  32  may be presented with all of the fragment results for all of the portions of the transcribed utterance at the same time.  FIG. 7  is a graphical depiction, on a transmitting device  12 , of the transcription of the utterance  36  of  FIG. 3 , illustrating the various alternatives reached by the ASR engine for each portion of the utterance  36 . The final transcription result, representing the result in which the ASR engine has the highest confidence level, is presented at the top. Other fragment results returned by the ASR engine for each of the various portions of the utterance  36  are presented beneath the highest-confidence result. More particularly, “batman” and “hat man” are presented as alternative fragment results for the utterance fragment that became “that man;” “engine ear,” “in here” and “ear” are presented as alternative fragment results for the utterance fragment that became “engineer;” “was a” and “was uh” are presented as alternative fragment results for the utterance fragment that became “is a;” “serial,” “see real” and “surreal” are presented as alternative fragment results for the utterance fragment that became “cereal;” and “killer,” “biller” and “fuller” are presented as alternative fragment results for the utterance fragment that became “filler.” 
         [0177]    In at least some embodiments, information about the order in which the ASR engine reached the results is provided in the form of animation or via other means, and such a process may be implemented, for example, in accordance with the disclosure of U.S. patent application Ser. No. 12/400,723, filed Mar. 9, 2009 and entitled “USE OF INTERMEDIATE SPEECH TRANSCRIPTION RESULTS IN EDITING FINAL SPEECH TRANSCRIPTION RESULTS,” which, together with any corresponding patent application publications thereof, is incorporated herein by reference. For example, incremental results may be displayed in an animated, real-time visual display, which then updates frequently as new information becomes available. In this way, the user  32  is exposed to most or all of the options that the ASR engine considered during transcription and can more easily navigate to those options, after transcription is complete, in order to select a transcription option different from the one chosen by the engine as having the highest confidence value. Various techniques for displaying real time information about preliminary and intermediate transcription results are described in the aforementioned patent application. 
         [0178]    Referring again to  FIG. 7 , it will be appreciated that each utterance fragment or portion, and its corresponding alternate transcriptions, may be a single word, or may be a phrase made up of multiple words. The drop-down lists, or any other graphical arrangement of alternate transcription fragments, can be displayed in an animated fashion after the entire transcription has been received. The lists of alternate transcription fragments can then be removed from the display leaving only the highest-confidence results visible. 
         [0179]    In accordance with one or more preferred embodiments of the present invention, the process described thus far may be further enhanced by providing certain identifying data, available to the user, to the ASR engine in order to make the conversion of an utterance to text more accurate. This data may be referred to as “metadata.” In the context of a voicemail, such metadata may include, for example, caller identification (“Caller ID”) (transmitting user) data such as the caller&#39;s name and the phone number corresponding to the calling device  12  (i.e., the source of the voicemail), recipient identification (“Recipient ID”) (receiving user) data such as the recipient&#39;s name and the phone number corresponding to the receiving device  14  (i.e., the recipient of the voicemail), address book data (data stored in the calling/transmitting or receiving device  12 , 14 ), and the like. 
         [0180]    The metadata may be used to more accurately transcribe certain utterance fragments, including names, companies, telephone numbers and the like. This may be very useful, for example, in the context of voicemails, which frequently include names and telephone numbers of callers, and, perhaps to a lesser extent, of recipients as well. For example, a typical voicemail may be generated by the following utterance  36 :
       “Hello Harry, this is Jim Berger, just wanted to let you know we are on for Tuesday. Please call me back at 704-555-1212 when you get a chance.”
 
However, a conventional ASR engine may transcribe such an utterance as follows:
   “Hello hairy, this is gym burgher, just wanted to let you know we are on for Tuesday. Please call me back at seven ero for five five five hun to hun to when you get a chance.”
 
As will be apparent, the conventional transcription of the utterance  36  includes inaccuracies pertaining to names and numbers included in the utterance  36 .
       
 
         [0183]      FIG. 8A  is a high-level flowchart illustrating the operation of a process  800  that utilizes metadata such as that described to produce a more accurate transcription of the utterance  36 . As shown therein, at least a portion of the process  800  begins at step  805  when a recorded utterance  36  is sent, together with metadata, to the ASR system  18  for transcription. The utterance  36  may be recorded at a calling or transmitting device  12 , the utterance  36  may be received at a receiving device  14  and then sent from the receiving device  14  to the ASR system  18  for transcription, or the utterance may be recorded at a calling device  12  and sent directly to the ASR system  18  for transcription and subsequent transmission as a text message from the calling device  12  to a receiving device  14 . In either case, metadata from the receiving device  14  or calling device  12 , respectively, is sent to the ASR system  18  along with the recorded utterance. 
         [0184]    At step  810 , the ASR engine transcribes the utterance  36  according to conventional transcription techniques. In particular, as part of the standard output produced by the ASR engine, an n-best list of all the utterance fragment possibilities and their corresponding statistical confidence score for each utterance fragment is created.  FIG. 9A  is a table listing various possible transcription results for each of several fragments or portions of the exemplary utterance  36  set forth above. Often, and particularly for proper nouns such as names and the like, the ASR engine may fail to select, as its final output, the proper result for every fragment of the utterance  36 . In many cases, however, the proper result may be included as one of the other possible results considered by the ASR engine. It will be appreciated that the chances that the correct result for a particular utterance fragment is included in the list of possible results are increased when the value of n is increased. Thus, although considered by the ASR engine, these results are not included in the final output because the confidence levels for the results were not as high as those in the final output. 
         [0185]    As described in one or more of the commonly-assigned patent applications referenced herein, the results thus created (i.e., results similar to those shown in the table of  FIG. 9A ) may, in at least some embodiments or applications, be returned to the user&#39;s device for manual editing or re-recording as desired. However, in at least some cases, the results may be improved through the use of the metadata supplied with the recorded utterance  36 . More particularly, at step  815 , each possible result for one or more fragment in the utterance  36  is compared to at least a portion of the metadata to determine if there is a statistically significant match. For metadata of the type described previously, the possible results may thus be compared to Caller ID data, Recipient ID data, address book data, and the like. It will be appreciated that the comparison may be done sequentially or in parallel. If at step  820  a statistically significant match is found, then at step  825 , if the possible result for which the match is found is not already in the final output, such possible result is promoted to the final output. 
         [0186]    In the exemplary utterance  36  set forth above, this comparison may take place as follows. First, the word “hello” and all its n-best list results are compared against the Caller ID/Recipient ID/address book data. Because the word “hello” is not the type of data found typically found in a Caller ID, Recipient ID or address book, it is unlikely that a statistically significant match will be found in the metadata and thus the result originally returned for this utterance fragment will likely remain unchanged. On the other hand, when the word “hairy” and all its n-best list results are compared against the Caller ID/Recipient ID/address book data, a match may be found, for example, with the Recipient ID (such as when the name of the user of the recipient device  14  is “Harry”), in which case the matched possible result in the n-best list is promoted to the top of the list. In other words, if the Recipient ID includes the name “Harry,” then the originally-proposed output “hairy” is replaced with the word or name “Harry” from the Recipient ID. Continuing through the originally-proposed transcription that was produced at step  810 , the words “this,” “just,” “wanted,” and the like remain unchanged, but comparisons of the words “gym” and “burgher” may find matches in Caller ID data (such as when the name of the user of the calling device  12  is “Jim Berger”), and thus the originally-proposed outputs “gym” and “burgher” are replaced with the words or names “Jim” and “Berger,” respectively, from the Caller ID data. Proper results for all proper nouns in the original output have now been provided using this method and the transcribed sentence would now read:
       “Hello Harry, this is Jim Berger, just wanted to let you know we are on for Tuesday. Please call me back at seven ero for five five five hun to hun to when you get a chance.”       
 
         [0188]    Similar techniques may also be combined with other filtering and processing techniques to achieve still further-improved results. For example, as described in the aforementioned U.S. patent application Ser. No. 12/198,112, possible results in the n-best list that are words representing particular numbers, or words that are homophones of words representing particular numbers, may be processed by a “Digit Filter” that substitutes Arabic numerals for those words. The use of such a filter is illustrated in  FIG. 8B  at step  827 . When each fragment in a sequence of utterance fragments may be replaced by an Arabic numeral, the corresponding sequence of Arabic numerals may be compared at step  830  to the metadata as described previously. If at step  835  a match is found, then at step  840  the entire sequence of utterance fragments is replaced with the sequence of Arabic numerals from the metadata. 
         [0189]    For example,  FIG. 9B  is a table listing various possible transcription results for each of several fragments or portions of the exemplary utterance  36  set forth above. After producing these results, the ASR engine or post-transcription filter may determine that alternative possible results for the word sequence “seven ero for five five five hun to hun to” includes the Arabic numeral sequence “7 0 4 5 5 5 1 2 1 2.” Further, a comparison of this Arabic numeral sequence to the Caller ID data identifies a match to the telephone number of the calling device. Thus, the originally-proposed output “seven ero for five five five hun to hun to” is replaced with the Arabic numeral sequence “7 0 4 5 5 5 1 2 1 2” from the Caller ID data. 
         [0190]    In at least some embodiments, the Arabic numeral sequence from the Caller ID data may be substituted for a sequence of utterance fragments in the originally-proposed result even when possible results in the n-best list do not include all of the Arabic numerals in the sequence. More particularly, a partial sequence of, for example, six of the seven Arabic numerals in a seven-digit telephone number included in the Caller ID data, or nine of the ten Arabic numerals in a ten-digit telephone number included in the Caller ID data, may still be replaced with the Arabic numerals of the telephone number. It will be appreciated that the parameters for invoking this technique may be varied as desired, and in fact may be set by a user. For example, the process may be limited to analyze the proposed results only for 7-digit sequences, only for 10-digit sequences, only for 11-digit sequences, for 6-, 7- and 8-digit sequences, for 9-, 10- and 11-digit sequences, for 10-, 11- and 12-digit sequences, or for any combination thereof, and such selection in some embodiments may be set by the operator of the system  10  while in other embodiments may be set by the user  32 , 34  of the calling or receiving device  12 , 14 . 
         [0191]    In at least some embodiments, a sequence of utterance fragments that is recognized as a telephone number may still further be passed through a “Telephone Filter” where hyphens are added to a sequence of 7, 10 or 11 digits to correctly format the digits into a telephone number. The use of such a filter is illustrated in  FIG. 8C  at step  837 . It will be appreciated that such a filter may be utilized whether or not there is a match between the number sequence output by the engine that was processed by the “Digit Filter” and the Caller ID or other metadata. If there is a metadata match, then the telephone number from the Caller ID data or other metadata is passed through the “Telephone Filter” to add hyphens and/or parentheses, and the result is substituted into the final proposed transcription result returned by the ASR system  18 . If there is no metadata match, then the results processed by the “Digit Filter” are kept, are passed through the “Telephone Filter” to add hyphens and/or parentheses, and the result is incorporated into the final proposed transcription result returned by the ASR system  18 . 
         [0192]    After the methods described herein are completed, the following final output after processing results in a correctly formatted and more accurate result:
       “Hello Harry, this is Jim Berger, just wanted to let you know we are on for Tuesday. Please call me back at 704-555-1212 when you get a chance.”
 
At step  845 , this result is then returned to the calling device  12  or recipient device  14 , respectively. It will be appreciated that the complete transcribed utterance resulting from the processes described hereinabove may be presented to, and edited by, a user as described and illustrated with respect to  FIGS. 6A-6E , at least with regard to text messages, and in corresponding manner with regard to voicemail and memo transcriptions.
       
 
         [0194]    It will be appreciated that the type and amount of metadata that is delivered to the ASR engine may be dependent on various factors. In implementations in which the ASR engine is located in the same device  12 , 14  that is making the transcription request, the metadata may be easily delivered and may include any data available to the device  12 , 14 , including complete address book data and the like. On the other hand, it will be further appreciated that modern client devices  12 , 14  may store or otherwise have available address book data and other metadata that is quite substantial in size. Thus, in order to minimize communication delays and the like, it may be desired to transmit only a portion of the available metadata to the ASR engine for use in the transcription process. Preferences may be given, for example, to metadata associated with the most recent telephone calls made or received, to a subset of the metadata from an address book (such as only one name, one phone number, and one company name, when a plurality of names, phone numbers and/or company names are available), or to any other more subset of the metadata available to the device  12 , 14 . Furthermore, some portion of the metadata may be stored in the ASR system  18  and updated on some appropriate basis, and then referenced when the device  12 , 14  associated with the metadata requests a transcription to occur. 
         [0195]    Furthermore, the system  10  may gain the ability to use information about edits and corrections made by the user  32  to update the language models (“LMs”) used by the ASR engine to produce the transcription, as described in U.S. patent application Ser. No. 12/407,502, filed Mar. 19, 2009, which is hereby incorporated herein by reference. 
         [0196]    It will be appreciated that automated transcription of recorded utterances  36  is useful in other environments and applications as well. For example, in another system (not separately illustrated), a user speaks an utterance  36  into a device as a voicemail, and the recorded speech audio is sent to the ASR system  18 . In another system, a system element that may or may not be associated directly with the ASR system  18  screens each word in a transcription for the availability of additional content associated with that word, and words for which content is available are enabled such that when one of the content-available words is spoken by the user, the ASR system  18  and/or other system elements recognize the spoken word and the additional content associated with that word is provided to the user, all in accordance with the disclosure of U.S. patent application Ser. No. 12/198,116, filed Aug. 25, 2008 and entitled “FACILITATING PRESENTATION BY MOBILE DEVICE OF ADDITIONAL CONTENT FOR A WORD OR PHRASE UPON UTTERANCE THEREOF,” which, together with any corresponding patent application publications thereof, is incorporated herein by reference. Other applications to which the teachings of the present invention are applicable will be apparent to the Ordinary Artisan. 
       Commercial Implementation  
       [0197]    One commercial implementation of the foregoing principles utilizes the Yap® and Yap9™ service (collectively, “the Yap service”), available from Yap Inc. of Charlotte, N.C. The Yap service includes one or more web applications and a client device application. The Yap web application is a J2EE application built using Java 5. It is designed to be deployed on an application server like IBM WebSphere Application Server or an equivalent J2EE application server. It is designed to be platform neutral, meaning the server hardware and OS can be anything supported by the web application server (e.g. Windows, Linux, MacOS X). 
         [0198]      FIG. 10  is a block diagram of the system architecture of the Yap commercial implementation. With reference to  FIG. 10 , the operating system may be implemented in Red Hat Enterprise Linux 5 (RHEL 5); the application servers may include the Websphere Application Server Community Edition (WAS-CE) servers, available from IBM; the web server may be an Apache server; the CTTS servlets may include CTTS servlets from Loquendo, including US/UK/ES male and US/UKIES female; the Grammar ASP may be the latest WebSphere Voice Server, available from IBM; suitable third party ads may be provided by Google; a suitable third party IM system is Google Talk, available from Google; and a suitable database system is the DB2 Express relational database system, available from IBM. 
         [0199]      FIG. 11  is a block diagram of the Yap EAR of  FIG. 10 . The audio codec JARs may include the VoiceAge AMR JAR, available from VoiceAge of Montreal, Quebec and/or the QCELP JAR, available from Qualcomm of San Diego, Calif. 
         [0200]    The Yap web application includes a plurality of servlets. As used herein, the term “servlet” refers to an object that receives a request and generates a response based on the request. Usually, a servlet is a small Java program that runs within a Web server. Servlets receive and respond to requests from Web clients, usually across HTTP and/or HTTPS, the HyperText Transfer Protocol. Currently, the Yap web application includes nine servlets: Correct, Debug, Install, Login, Notify, Ping, Results, Submit, and TTS. Each servlet is described below in the order typically encountered. 
         [0201]    The communication protocol used for all messages between the Yap client and Yap server applications is HTTP and HTTPS. Using these standard web protocols allows the Yap web application to fit well in a web application container. From the application server&#39;s point of view, it cannot distinguish between the Yap client midlet and a typical web browser. This aspect of the design is intentional to convince the web application server that the Yap client midlet is actually a web browser. This allows a user to use features of the J2EE web programming model like session management and HTTPS security. It is also an important feature of the client as the MIDP specification requires that clients are allowed to communicate over HTTP. 
         [0202]    More specifically, the Yap client uses the POST method and custom headers to pass values to the server. The body of the HTTP message in most cases is irrelevant with the exception of when the client submits audio data to the server in which case the body contains the binary audio data. The Server responds with an HTTP code indicating the success or failure of the request and data in the body which corresponds to the request being made. Preferably, the server does not depend on custom header messages being delivered to the client as the carriers can, and usually do, strip out unknown header values.  FIG. 12  is a typical header section of an HTTP request from the Yap client. 
         [0203]    The Yap client is operated via a user interface (UT), known as “Yap9,” which is well suited for implementing methods of converting an audio message into a text message and messaging in mobile environments. Yap9 is a combined UT for SMS and web services (WS) that makes use of the buttons or keys of the client device by assigning a function to each button (sometimes referred to as a “Yap9” button or key). Execution of such functions is carried out by “Yaplets.” This process, and the usage of such buttons, are described elsewhere herein and, in particular, in  FIGS. 10A-10D , and accompanying text, of the aforementioned U.S. Patent Application Pub. No. US 2007/0239837. 
         [0204]    Usage Process—Install: Installation of the Yap client device application is described in the aforementioned U.S. Patent Application Pub. No. US 2007/0239837 in a subsection titled “Install Process” of a section titled “System Architecture.” 
         [0205]    Usage Process—Notify: When a Yap client is installed, the install fails, or the install is canceled by the user, the Notify servlet is sent a message by the phone with a short description. This can be used for tracking purposes and to help diagnose any install problems. 
         [0206]    Usage Process—Login: When the Yap midlet is opened, the first step is to create a new session by logging into the Yap web application using the Login servlet. Preferably, however, multiple login servers exist, so as a preliminary step, a request is sent to find a server to log in to. Exemplary protocol details for such a request can be seen in  FIG. 13 . An HTTP string pointing to a selected login server will be returned in response to this request. It will be appreciated that this selection process functions as a poor man&#39;s load balancer. 
         [0207]    After receiving this response, a login request is sent. Exemplary protocol details for such a request can be seen in  FIG. 14 . A cookie holding a session ID is returned in response to this request. The session ID is a pointer to a session object on the server which holds the state of the session. This session data will be discarded after a period determined by server policy. 
         [0208]    Sessions are typically maintained using client-side cookies, however, a user cannot rely on the set-cookie header successfully returning to the Yap client because the carrier may remove that header from the HTTP response. The solution to this problem is to use the technique of URL rewriting. To do this, the session ID is extracted from the session API, which is returned to the client in the body of the response. This is called the “Yap Cookie” and is used in every subsequent request from the client. The Yap Cookie looks like this:
       ;jsessionid=C240B217F2351E3C420A599B0878371A       
 
         [0210]    All requests from the client simply append this cookie to the end of each request and the session is maintained:
       /Yap/Submit;jsessionid=C240B217F2351E3C420A599B0878371A       
 
         [0212]    Usage Process—Submit: After receiving a session ID, audio data may be submitted. The user presses and holds one of the Yap-9 buttons, speaks aloud, and releases the pressed button. The speech is recorded, and the recorded speech is then sent in the body of a request to the Submit servlet, which returns a unique receipt that the client can use later to identify this utterance. Exemplary protocol details for such a request can be seen in  FIG. 15 . 
         [0213]    One of the header values sent to the server during the login process is the format in which the device records. That value is stored in the session so the Submit servlet knows how to convert the audio into a format required by the ASR engine. This is done in a separate thread as the process can take some time to complete. 
         [0214]    The Yap9 button and Yap9 screen numbers are passed to the Submit server in the HTTP request header. These values are used to lookup a user-defined preference of what each button is assigned to. For example, the 1 button may be used to transcribe audio for an SMS message, while the 2 button is designated for a grammar based recognition to be used in a web services location based search. The Submit servlet determines the appropriate “Yaplet” to use. When the engine has finished transcribing the audio or matching it against a grammar, the results are stored in a hash table in the session. 
         [0215]    In the case of transcribed audio for an SMS text message, a number of filters can be applied to the text returned from the ASR engine. Such filters may include, but are not limited to, those shown Table 1. 
         [0000]                        TABLE 1               Filter Type   Function                   Ad Filter   Used to scan the text and identify keywords that           can be used to insert targeted advertising           messages, and/or convert the keywords into           hyperlinks to ad sponsored web pages       Currency   Used to format currency returned from the speech       Filter   engine into the user&#39;s preferred format. (e.g.,           “one hundred twenty dollars” -&gt; “$120.00”)       Date Filter   Used to format dates returned from the speech           engine into the user&#39;s preferred format. (e.g.,           “march fourth two thousand seven” -&gt; “3/4/2007”)       Digit Filter   User to format spelled out single digits returned           from the speech engine into a multi-digit number           such as a zip code (e.g., “two eight two one one”           -&gt; “28211”)       Engine   Used to remove speech engine words       Filter       Number   Used to convert the spelled out numbers returned       Filter   from the speech engine into a digit based number           (e.g., “one hundred forty seven” -&gt; “147”)       Obscenity   Used to place asterisks in for the vowels in       Filter   street slang (e.g., “sh*t”, “f*ck”, etc.)       Punctuation   Used to format punctuation       Filter       SMS Filter   Used to convert regular words into a spelling           which more closely resembles an SMS message (e.g.,           “don&#39;t forget to smile” -&gt; “don&#39;t 4get 2 :)”,           etc.)       Time Filter   Used to format time phrases                    
Notably, after all of the filters are applied, both the filtered text and original text are returned to the client so that if text to speech is enabled for the user, the original unfiltered text can be used to generate the TTS audio.
 
         [0216]    Usage Process—Results: The client retrieves the results of the audio by taking the receipt returned from the Submit servlet and submitting it as a request to the Results servlet. Exemplary protocol details for such a request can be seen in  FIG. 16 . This is done in a separate thread on the device and a timeout parameter may be specified which will cause the request to return after a certain amount of time if the results are not available. In response to the request, a block of XML is preferably returned. Exemplary protocol details for such a return response can be seen in  FIG. 17 . Alternatively, a serialized Java Results object may be returned. This object contains a number of getter functions for the client to extract the type of results screen to advance to (i.e., SMS or results list), the text to display, the text to be used for TTS, any advertising text to be displayed, an SMS trailer to append to the SMS message, etc. 
         [0217]    Usage Process—TTS: The user may choose to have the results read back via Text to Speech. This can be an option the user could disable to save network bandwidth, but adds value when in a situation where looking at the screen is not desirable, like when driving. If TTS is used, the TTS string is extracted from the results and sent via an HTTP request to the TTS servlet. Exemplary protocol details for such a request can be seen in  FIG. 18 . The request blocks until the TTS is generated and returns audio in the format supported by the phone in the body of the result. This is performed in a separate thread on the device since the transaction may take some time to complete. The resulting audio is then played to the user through the AudioService object on the client. Preferably, TTS speech from the server is encrypted using Corrected Block Tiny Encryption Algorithm (XXTEA) encryption. 
         [0218]    Usage Process—Correct: As a means of tracking accuracy and improving future SMS based language models, if the user makes a correction to transcribed text on the phone via the key elements of the key input unit  74  before sending the message, the corrected text is submitted to the Correct servlet along with the receipt for the request. This information is stored on the server for later use in analyzing accuracy and compiling a database of typical SMS messages. Exemplary protocol details for such a submission can be seen in  FIG. 19 . 
         [0219]    Usage Process—Ping: Typically, web sessions will timeout after a certain amount of inactivity. The Ping servlet can be used to send a quick message from the client to keep the session alive. Exemplary protocol details for such a message can be seen in  FIG. 20 . 
         [0220]    Usage Process—Debug: Used mainly for development purposes, the Debug servlet sends logging messages from the client to a debug log on the server. Exemplary protocol details can be seen in  FIG. 21 . 
         [0221]    Usage Process—Logout: To logout from the Yap server, an HTTP logout request needs to be issued to the server. An exemplary such request would take the form: “/Yap/Logout;jsessionid=1234”, where 1234 is the session ID. 
         [0222]    User Preferences: In at least one embodiment, the Yap website has a section where the user can log in and customize their Yap client preferences. This allows them to choose from available Yaplets and assign them to Yap9 keys on their phone. The user preferences are stored and maintained on the server and accessible from the Yap web application. This frees the Yap client from having to know about all of the different back-end Yaplets. It just records the audio, submits it to the server along with the Yap9 key and Yap9 screen used for the recording and waits for the results. The server handles all of the details of what the user actually wants to have happen with the audio. 
         [0223]    The client needs to know what type of format to utilize when presenting the results to the user. This is accomplished through a code in the Results object. The majority of requests fall into one of two categories: sending an SMS message, or displaying the results of a web services query in a list format. Notably, although these two are the most common, the Yap architecture supports the addition of new formats. 
         [0224]    Based on the foregoing description, it will be readily understood by those persons skilled in the art that the present invention is susceptible of broad utility and application. Many embodiments and adaptations of the present invention other than those specifically described herein, as well as many variations, modifications, and equivalent arrangements, will be apparent from or reasonably suggested by the present invention and the foregoing descriptions thereof, without departing from the substance or scope of the present invention. 
         [0225]    Accordingly, while the present invention has been described herein in detail in relation to one or more preferred embodiments, it is to be understood that this disclosure is only illustrative and exemplary of the present invention and is made merely for the purpose of providing a full and enabling disclosure of the invention. The foregoing disclosure is not intended to be construed to limit the present invention or otherwise exclude any such other embodiments, adaptations, variations, modifications or equivalent arrangements, the present invention being limited only by the claims appended hereto and the equivalents thereof.