Patent Publication Number: US-6707891-B1

Title: Method and system for voice electronic mail

Description:
FIELD OF THE INVENTION 
     The present invention relates to management of electronic mail. 
     BACKGROUND OF THE INVENTION 
     Electronic mail, or e-mail, is fast becoming a dominant mode of communication. In its fundamental form, e-mail is managed by two kinds of computer software; namely, e-mail server software (referred to simply as “e-mail servers”) and e-mail client software (referred to simply as “e-mail clients”). E-mail servers reside on centrally accessed computers (referred to as “server computers”), and manage incoming and outgoing e-mail for remote users who are authorized to use such server computers. E-mail servers maintain separate mail accounts for each authorized user, into which incoming mail is stored. 
     E-mail clients reside on remote computers (referred to as “client computers”) and manage incoming and outgoing mail for designated users. E-mail clients are responsible for downloading arriving mail items from the server computer at which they are located. E-mail clients are also responsible for transmitting outgoing mail from a remote client computer to a server computer. Examples of popular e-mail clients for personal computers are Eudora®, Internet Explorer® and Netscape®. 
     Standard e-mail uses a protocol named Simple Mail Transport Protocol (SMTP). For use over the Internet, Multi-Purpose Internet Mail Extension (MIME) extends the format of Internet mail to allow non-US-ASCII textual messages, non-textual messages, multi-part message bodies and non-US-ASCII information in message headers. Attachments to e-mail messages are also encoded as MIME attachments. 
     One of the drawbacks with e-mail, however, is the need for a user to be connected to the Internet in order to receive or send his e-mail. Although Internet access is available in many homes and offices, and Internet stations are available in many public areas, being “un-wired” is still a concern for mobile users who are neither at home nor in their office. When on the road, a user often finds himself unable to access his e-mail, for a variety of reasons. 
     SUMMARY OF THE INVENTION 
     There is thus provided in accordance with a preferred embodiment of the present invention a method for management of electronic mail, including the steps of converting an e-mail message from text to speech, receiving an input request for a selected e-mail message, reading the selected e-mail message, recording a reply to the selected e-mail message, producing an audio file, and sending the audio file as an attachment to a reply e-mail. 
     There is also provided in accordance with a preferred embodiment of the present invention a method for management of electronic mail, including the steps of converting an e-mail message from text to speech, receiving an input request for a selected e-mail message, reading the selected e-mail message, recording a reply to the selected e-mail message, producing an audio file, storing the audio file on a computer, and sending a reply e-mail containing a link to the audio file. 
     There is also provided in accordance with a preferred embodiment of the present invention a system for management of electronic mail, including a text-to-speech converter converting an e-mail message from text to speech, a receiver receiving an input request for a selected e-mail message, an audio player reading the selected e-mail message, an audio recorder recording a reply to the selected e-mail message, producing an audio file, and a transmitter sending the audio file as an attachment to a reply e-mail. 
     There is also provided in accordance with a preferred embodiment of the present invention a system for management of electronic mail, including a text-to-speech converter converting an e-mail message from text to speech, a receiver receiving an input request for a selected e-mail message, an audio player reading the selected e-mail message, an audio recorder recording a reply to the selected e-mail message, producing an audio file, a computer storing the audio file, and a transmitter sending a reply e-mail containing a link to the audio file. 
     There is also provided in accordance with a preferred embodiment of the present invention a method for managing voice electronic mail including the steps of playing by a local computer an incoming audio file containing a voice message, the incoming audio file residing on a remote computer, and saving the incoming audio file as a local audio file on the local computer after the playing step. 
     There is also provided in accordance with a preferred embodiment of the present invention a method for management of electronic mail, including the steps of converting an e-mail message from text to speech, receiving an input request for a selected e-mail message, reading the selected e-mail message, recording a reply to the selected e-mail message, producing an audio file, sending the audio file as an attachment to a reply e-mail, and playing the audio file. 
     There is also provided in accordance with a preferred embodiment of the present invention a method for management of electronic mail, including the steps of converting an e-mail message from text to speech, receiving an input request for a selected e-mail message, reading the selected e-mail message, recording a reply to the selected e-mail message, producing an audio file, storing the audio file on a first computer, sending a reply e-mail containing a link to the audio file, playing the audio file, and saving the audio file. 
     There is also provided in accordance with a preferred embodiment of the present invention a system for managing voice electronic mail including an audio player within a local computer playing an incoming audio file containing a voice message, the incoming audio file residing on a remote computer, and a data processor saving the incoming audio file as a local audio file on the local computer, after the audio player plays the incoming audio file. 
     There is also provided in accordance with a preferred embodiment of the present invention a system for management of electronic mail, including a text-to-speech converter converting an e-mail message from text to speech, a receiver receiving an input request for a selected e-mail message, a first audio player reading the selected e-mail message, an audio recorder recording a reply to the selected e-mail message, producing an audio file, a transmitter sending the audio file as an attachment to a reply e-mail, and a second audio player playing the audio file. 
     There is also provided in accordance with a preferred embodiment of the present invention a system for management of electronic mail, including a text-to-speech converter converting an e-mail message from text to speech, a receiver receiving an input request for a selected e-mail message, a first audio player reading the selected e-mail message, an audio recorder recording a reply to the selected e-mail message, producing an audio file, a computer storing the audio file, a transmitter sending a reply e-mail containing a link to the audio file, a second audio player playing the audio file, and a data processor saving the audio file. 
    
    
     BRIEF DESCRIPTION OF THE DRAWINGS 
     The present invention will be more fully understood and appreciated from the following detailed description, taken in conjunction with the drawings in which: 
     FIG. 1 is a simplified illustration of an e-mail management system in accordance with a preferred embodiment of the present invention. 
    
    
     DETAILED DESCRIPTION OF A PREFERRED EMBODIMENT 
     The present specification describes a method and system for retrieving e-mail by dialing in from a telephone to a voice response unit. The e-mail is converted to speech and read to the user over the phone. Various selections can be made by pushing the buttons of the telephone, and the user can effectively scroll through his e-mail and choose the ones to listen to. 
     The present specification also describes a method and system for replying to e-mail through a telephone. The reply takes one or both of two forms: (1) a text e-mail reply, generated from the speech reply by a speech-to-text converter, and (2) an audio file that contains a recording of the speech reply therewithin. In one preferred embodiment of the present invention, the audio file is sent back to the original sender as a MIME attachment within a reply e-mail, in a standard audio format such as a WAV file. When the original sender receives the reply e-mail, he can download the WAV file to his local computer and play the file to listen to the recorded reply. 
     In another preferred embodiment the audio file is posted on an Internet web site in a streaming audio format such as RealAudio®, and a reply e-mail notification is sent to the original sender having an Internet link to the audio file therewithin. When the original sender receives the reply e-mail, he can click on the link to play the audio file. Streaming audio formats enable the user to play an audio file on-line, while it is being delivered, rather than have to wait until the entire audio file is downloaded. REALAUDIO is a registered trademark of Progressive Networks, Inc. Information about RealAudio files is available on the Internet at http://www.realaudio.com. 
     Advantages of the latter embodiment are (1) the e-mail reply is small in size, since it only contains a link to the audio file, rather than the file itself; and (2) the audio file is streamed for on-line playing. An advantage of the former embodiment is that the reply e-mail is self-contained, and the user does not have to use a web browser to listen to the voice reply. 
     It should be pointed out that from an ergonometric point of view, both embodiments have the advantage that e-mail messages are being replied to with speech, and the replies are being listened to rather than read. Dictating e-mail messages into a voice response unit over a telephone line, by means of a speech-to-text converter, is faster and simpler than typing a text message. For users who receive large volumes of e-mail on a daily basis, reading through e-mail is both time-consuming, and difficult on one&#39;s eyes. Many professionals spend hours daily reading and replying to e-mail messages. By listening to e-mail, rather than reading it, these professionals can work on other “vision-intensive” tasks while they are listening to their e-mail messages at the same time. This is analogous to working on a task while listening in on a conference call being played on the speaker of a telephone. 
     Reference is now made to FIG. 1, which is a simplified illustration of an e-mail management system in accordance with a preferred embodiment of the present invention. 
     To assist in understanding the present invention, a simplified example session is described. 
     EXAMPLE SESSION 
     As a simplified example of the operation of a preferred embodiment of the present invention, a mobile user dials in to a voice response unit (VRU) by means of a telephone to access his e-mail remotely. The VRU reads out a message indicating that the user has e-mail in his inbox. Each e-mail item contains a unique identifier, data fields including FROM, RE and DATE, a text body, and one or more attachments. Each e-mail message is converted from text to speech by a text-to-speech converter within the VRU. 
     The VRU reads out a first menu of options as follows: 
     1. ID:018, FROM:ANDY, RE:MEETING TOMORROW, DATE:mm/dd/yy 
     2. ID:004, FROM:BILL, RE:SUNDAY PICNIC, DATE:mm/dd/yy 
     3. ID:103, FROM:CHARLIE, RE:UPCOMING EXAM, DATE:mm/dd/yy 
     4. ID:075, FROM:DAN, RE:DOCTOR&#39;S APPOINTMENT, DATE:mm/dd/yy 
     5. &lt;MORE&gt; 
     Option 5, “&lt;MORE&gt;”, is used for menus that entail more information than is presented in a single list. By selecting “&lt;MORE&gt;”, the user requests additional lists of menu items. Correspondingly, successive lists could have a “&lt;BACK&gt;” option for returning to previous lists. 
     After listening to a specific e-mail message, the user can reply to the message and/or route the message to his fax machine or to his pager. For reply purposes, the user speaks into his telephone and the VRU records the speech. The VRU then reads a second menu of options as follows: 
     1. POST REPLY AS A REALAUDIO FILE AND SENT NOTIFICATION WITH LINK 
     2. ATTACH REPLY AS A WAV FILE TO E-MAIL REPLY 
     3. BOTH 
     Selecting option 1 instructs the VRU to convert the speech reply to a RealAudio file and post it on an Internet web site, and to send an e-mail reply notification containing a link to the RealAudio file. Selecting option 2 instructs the VRU to convert the speech reply to a WAV file and attach the WAV file to an e-mail reply. Selecting option 3 instructs the VRU to post the reply and also send it as an e-mail attachment. In this latter case, the receiver has a choice of whether to access the reply message from the attached WAV file or from the Internet site. 
     Alternatively, in a preferred embodiment of the present invention, the choice between a RealAudio file and a WAV file can be pre-programmed, thus avoiding the need for the second menu of options hereinabove. The pre-programmed choice of audio file can be customized for each reply address. 
     For routing purposes, the VRU reads a third menu of options as follows: 
     1. FAX E-MAIL TO ME 
     2. FAX E-MAIL ATTACHMENT TO ME 
     3. SEND TEXT ONLY TO MY PAGER 
     4. &lt;MORE&gt; 
     Selecting option 1 instructs the VRU to fax e-mail, without attachments, to the user&#39;s fax machine. Selecting option 2 instructs the VRU to fax the e-mail attachment(s) to the user&#39;s fax machine. Selecting option 3 instructs the VRU to send the text body of the e-mail to the user&#39;s pager. 
     Reference is now made to FIG. 1, which illustrates an e-mail management system in accordance with a preferred embodiment of the present invention. An e-mail server  110  receives e-mail designated for a specific user account. A voice response unit (VRU)  120  downloads the user&#39;s e-mail via an e-mail client  130 , and converts each e-mail message from text to speech via a text-to-speech converter  140 . 
     In a preferred embodiment of the present invention, the user dials in to the VRU from telephone  150  to retrieve his e-mail. By pressing on the telephone pad in accordance with menu options read from the VRU, the user selects e-mail messages to be read to him by the VRU. After listening to an e-mail message, the user replies into the telephone, and the VRU records the reply. The user then instructs the VRU, by means of the telephone pad, to convert the recorded reply into an audio file, such as a WAV file, and send it back to the original sender&#39;s e-mail account  160  as an attachment to a reply e-mail. Alternatively, the user can instruct the VRU to convert the recorded reply into a streaming audio format, such as a RealAudio format, and post it on an Internet web site  170 . In this latter case, the VRU also sends an e-mail notification to the original sender&#39;s e-mail account  160 , containing a link to the reply audio file on Internet web site  170 . 
     From the sender&#39;s e-mail account  160 , the mail can be retrieved by the sender&#39;s computer  180 , by the sender&#39;s pager  190 , or by the sender dialing in from telephone  200 . Alternatively, the sender may also be an authorized user of a voice response unit similar to voice response unit  120 , and, as such, he can dial in by telephone to retrieve his e-mail. 
     In a preferred embodiment of the present invention, the user can also instruct the VRU to route his incoming e-mail message by sending it as a text file to his own e-mail account  210 , or to his fax machine  220 . From the user&#39;s e-mail account, the mail can be retrieved by the user&#39;s computer  230  or by the user&#39;s pager  240 . 
     In an alternative embodiment of the present invention the audio file attached to a reply e-mail can be compressed as a self-extracting compressed file. Self-extracting files contain decompressors within them. The receiver of an e-mail message with a self-extracting file attached therewithin clicks on the file, and the compressed audio file automatically decompresses itself. For example, the present invention can attach a zipped WAV file to a reply e-mail. The attachment would automatically unzip itself and generate the WAV file when a receiver clicks on the link to the attachment. In this alternative embodiment the VRU also contains an encoder  250  for compressing audio files. 
     Although it is the “un-wired” mobile user who typically accesses his e-mail through a telephone unit, the present invention is also advantageous to the “wired” user who accesses his e-mail through a computer connected to a network. For reasons of convenience, and other reasons as well, a user with a computer may nevertheless prefer to reply to e-mail messages and send new e-mail messages using speech rather than text. 
     The present invention also provides an audio e-mail client that manages incoming and outgoing audio messages. It enables a user to dictate a voice e-mail message by speaking into a microphone connected to his computer, and convert the speech into an audio file. The audio file can then be either (1) sent as an attachment to a standard e-mail message, or (2) posted on an Internet web site. In the latter case an e-mail notification message with a link to the audio file is sent. It can thus be appreciated that the present invention provides telephone-like capability to a computer user. 
     The audio e-mail client of the present invention enables a user to save an audio file that is in a streaming audio format on his local computer. This feature is not available in standard RealAudio players. Such players are used for listening to live broadcasts over the Internet, such as newscasts, and as such, are typically used to play an audio file once. The present invention introduces a novel use of streaming audio files; namely, for voice e-mail messages, and in this context it is important to be able to save the voice messages for future reference. 
     In a preferred embodiment, the audio e-mail client of the present invention provides a mail archive for voice messages that is structured hierarchically in files and directories, as is common today for text e-mail messages. When an incoming voice e-mail message is saved, the user can direct it to a specific directory or sub-directory, for cataloging and indexing purposes. Moreover the incoming voice e-mail message can be saved in a variety of formats, such as RealAudio, WAV or compressed WAV. 
     Referring back to FIG. 1, in a preferred embodiment of the present invention the sender may use such an audio e-mail client  260  to play the RealAudio file posted at Internet web site  170 , and also save the file on the sender&#39;s computer  180 . 
     In a preferred embodiment, the audio e-mail client of the present invention also carries out remote functions by posting voice e-mail messages on Internet web site  170 . Specifically, a user can post voice e-mail messages on Internet web site for retrieval by another person, or for purposes of archiving his own voice e-mail. Audio e-mail client  260  has the capability to manage hard disk space on remote computers, by using an appropriate protocol for uploading and storing RealAudio and other audio files at Internet web site  170 . This is indicated in FIG. 1 by two directional arrows between Internet web site  170  and audio e-mail client  260 . 
     In a preferred embodiment of the present invention, Internet web site  170  resides in a server computer (not shown) that is part of an overall electronic mail management system. The overall management system also includes VRU  120  and, optionally, e-mail server  110 . The protocol for uploading and storing files on Internet web site  170  may be proprietary, and, if so, audio e-mail client  260  sends files and instructions to the server computer in accordance with such proprietary protocol. 
     As mentioned hereinabove, the present invention provides a capability for producing text e-mail messages from speech. To this end, VRU  120  also contains a speech-to-text converter  270 , for producing a text message from dictated speech into telephone  150 . 
     The present invention is also useful within a “buddy system.” Such a system is based upon a “buddy list,” which is a list of friends, or members, who wish to be notified when any members of the list are on-line. An example of such a system is the Mirabilis ICQ (“I seek you”) system for notifying members when a member of the list is connected to the Internet. Reference information on the ICQ system is available on the Internet at http://www.icq.com. Within the context of the present invention, VRU  120  can be programmed to notify members of a “buddy list” whenever a user who is one of the members dials in to his VRU. Upon notification the members are alerted that the user is currently dialed in to his VRU, and, as such, can be contacted therefrom. 
     It will be appreciated by persons skilled in the art that the present invention is not limited by what has been particularly shown and described hereinabove. Rather the present invention includes combinations and sub-combinations of the various features described hereinabove as well as modifications and extensions thereof which would occur to a person skilled in the art and which do not fall within the prior art.