Patent Document

CROSS-REFERENCE TO RELATED APPLICATIONS  
       [0001]     This application claims the benefit under 35 U.S.C. §119(e) of U.S. Provisional Application No. 60/592,561, filed Jul. 30, 2004.  
       REFERENCE TO COMPUTER PROGRAM LISTING/TABLE APPENDIX  
       [0002]     The present application includes a computer program listing appendix on compact disc. Two duplicate compact discs are provided herewith. Each compact disc contains an ASCII text file of the computer program listing as follows:  
                                       Name   Size   Created                   Category_Complaintsfrm_txt.txt   34.72 KB   07/30/2004               5:05:52 PM       Category_Contactsfrm_txt.txt   31.73 KB   07/30/2004               5:05:52 PM       Category_food_breakfastfrm_txt.txt   38.15 KB   07/30/2004               5:05:52 PM       Category_food_dinnerfrm_txt.txt   38.04 KB   07/30/2004               5:05:52 PM       Category_food_lunchfrm_txt.txt   37.95 KB   07/30/2004               5:05:52 PM       Category_food_snacksfrm_txt.txt   35.31 KB   07/30/2004               5:05:52 PM       Category_greetingsfrm_txt.txt   37.31 KB   07/30/2004               5:05:52 PM       Category_hobbiesfrm_txt.txt   26.38 KB   07/30/2004               5:05:52 PM       Category_needsfrm_txt.txt   31.89 KB   07/30/2004               5:05:52 PM       Category_relaxfrm_txt.txt   37.28 KB   07/30/2004               5:05:52 PM       Category_weatherfrm_txt.txt   34.85 KB   07/30/2004               5:05:52 PM       Categoryfrm_txt.txt   24.81 KB   07/30/2004               5:05:52 PM       David_Keyboardfrm_txt.txt   165.05 KB    07/30/2004               5:05:52 PM       Health_Carefrm_txt.txt   40.55 KB   07/30/2004               5:05:52 PM       Historyfrm_txt.txt   7.25 KB   07/30/2004               5:05:52 PM       Keyboard1frm_txt.txt   77.17 KB   07/30/2004               5:05:52 PM       Keyboard2frm_txt.txt   72.76 KB   07/30/2004               5:05:52 PM       Logger_Formfrm_txt.txt    3.23 KB   07/30/2004               5:05:52 PM       Module1bas_txt.txt    9.58 KB   07/30/2004               5:05:53 PM       Notepad_formfrm_txt.txt   11.22 KB   07/30/2004               5:05:53 PM       Personal_Infofrm_txt.txt   34.11 KB   07/30/2004               5:05:53 PM       SimpleTTSfrm_txt.txt   60.66 KB   07/30/2004               5:05:53 PM       Talker-2_July-02-2004vbp_txt.txt    1.93 KB   07/30/2004               5:05:53 PM                  
 
 The computer program listing appendix is hereby expressly incorporated by reference in the present application. 
 
     
    
     FIELD OF THE INVENTION  
       [0003]     The present invention relates generally to a speech output device and more specifically to an augmentative communications software program for the speech impaired having gross and fine motor impairments, which uses commercial-grade technology.  
       BACKGROUND OF THE INVENTION  
       [0004]     Speech output devices are well known in the art for aiding the speech impaired. Known devices typically comprise a computer software database with preprogrammed words, phrases, and sentences, and, at least one synthesized voice. Some devices further comprise a plurality of icons that when selected, speak a word relevant to the icon. Still others include a database for storing phrases and sentences that can be grouped according to the appropriate category for later use. These inventions, however, are costly and are difficult to use by those having gross and fine motor impairments. Hence, these devices provide for slow conversations since they do not offer the flexibility and efficiency desired when communicating.  
         [0005]     Thus, there is a longfelt need for a speech output device that provides a speech-impaired user with the ability to have flexible and efficient communications.  
       SUMMARY OF THE INVENTION  
       [0006]     The present invention broadly comprises a computer-based method for aiding aphasics having gross and fine motor impairments in efficiently communicating, comprising the steps of storing alphanumeric characters in a database, calculating statistics of the alphanumeric characters based on frequency used and most recent used, and, predicting a response according to the statistics of the alphanumeric characters, wherein the steps of storing, calculating, and predicting are performed by a general purpose computer specially programmed to perform the steps of storing, calculating, and predicting.  
         [0007]     It is therefore, an object of the present invention to provide a speech output device that has a predictive capability wherein the likelihood of a user&#39;s response is statistically calculated and presented according to words, phrases, and sentences entered or selected previously in relation to the frequency used, most recent used, time of day used, day of week used, and time of year used.  
         [0008]     Another object of the present invention is to provide preprogrammed phrases in a context-relevant manner.  
         [0009]     Another object of the present invention is to reduce the cost of an augmentative communications device.  
         [0010]     It is a further object of the present invention to provide a software program that allows a user to pre-enter a conversation for future use.  
         [0011]     These and other objects, features, and advantages of the present invention will become readily apparent to those having ordinary skill in the art upon reading the following detailed description of the invention in view of the several drawings of the invention. 
     
    
     BRIEF DESCRIPTION OF THE DRAWINGS  
       [0012]     The nature and mode of operation of the present invention will now be more fully described in the following detailed description of the invention taken with the accompanying drawing figures, in which:  
         [0013]      FIG. 1  is a photograph of the Normal Keyboard Menu screen of the present invention;  
         [0014]      FIG. 2  is a photograph of the Categories Menu screen of the present invention;  
         [0015]      FIG. 3  is a photograph of the Breakfast Menu screen, which is accessed by selecting the Breakfast button of the Categories Menu screen of  FIG. 2 ;  
         [0016]      FIG. 4  is a photograph of the Greetings screen, which is accessed by selecting the Greetings button of the Categories Menu screen of  FIG. 2 ;  
         [0017]      FIG. 5  is a photograph of the Special Keyboard for David screen, which is accessed by selecting the Keyboard button of the Categories Menu screen of  FIG. 2 ; and,  
         [0018]      FIG. 6  is a photograph of the Notepad screen of the present invention, which is accessed by selecting the Notepad button of  FIGS. 1-5 . 
     
    
     DETAILED DESCRIPTION OF THE INVENTION  
       [0019]     The invention is best understood from the perspective of an individual using the speech output device to communicate, illustrated via photographs from the software application of the present invention.  
         [0020]     The present invention broadly comprises a computerized method and apparatus for helping the speech impaired having means for converting words, phrases, and sentences such as a text-to-speech converter driven by a graphical user interface of a computer, means for inputting and displaying words, phrases, and sentences, and, a database having a predictive capability, and an applications programming interface. By “predictive capability” it is meant that words, phrases, and sentences are displayed by a computer in anticipation of a user&#39;s response, wherein a database stores words, phrases, and sentences and a computer statistically calculates and predicts which words, phrases, and sentences are likely to be used according to frequency and most recent used, time of day used, day of week used, and time of year used (i.e., seasonal). The predictive capability is provided by tagging words, phrases, and sentences according to frequency used, most recent used, time of day used, day of week used, and time of year used, and storing the tagged words, phrases, and sentences in a database such that future words, phrases, and sentences are predicted and presented to a user. As described in detail infra, the means for inputting and displaying the words, phrases, and sentences generally comprise an input device and a computer screen, respectively.  
         [0021]     The invention is now described with reference to photographs in the form of drawing figures arranged from the perspective of a user. The photographs, together with the written description of them, enable a person having ordinary skill in the art to use the invention, whereas the enclosed software code listing enables a person having ordinary skill in the art to make the invention. For purposes of illustration, a fictional name of a user appears throughout the drawing figures and specification.  
         [0022]     Adverting now to the figures,  FIG. 1  is a photograph of Normal Keyboard Menu screen  10  of the present invention. As shown in  FIG. 1 , graphical user interface comprises Normal Keyboard Menu screen  10 , which is first encountered upon entering the system However, it should be appreciated that the first screen encountered upon entering the system may be a predicted menu screen based on menu screens previously chosen by a user according to the time of day, day of week, and time of year and according to menu screens used most frequently and used most recently. In a preferred embodiment, the device is readily available on a laptop computer having a touch-screen, audio output via onboard speakers, and an input device such as a keyboard overlay, joystick, trackball, push-button, or eye movement input device. Another example of an input device that can be used is a brain wave sensing device allowing a user to send signals to a computer via a brain wave sensing headband. However, it should be readily apparent that other input devices may be used, and these modifications are intended to be within the spirit and scope of the invention as claimed. The device may be used on a desktop computer or adapted for use on a personal digital assistant (PDA) as well. Normal Keyboard Menu screen  10  comprises Keyboard  11 , Swap Keyboard buttons  12 , Notepad button  13 , Help button  14 , History button  15 , Categories button  16 , Keyboard button  18 , View screen  19 , Clear button  20 , Matching Sentences window  22 , windows  24 ,  25 ,  26 , and Speak buttons  23 ,  27 . It should be appreciated by those having ordinary skill in the art that the layout of the graphical user interface is intended to minimize the radial distance that the user has to move an input device to make a selection. For this reason, Normal Keyboard Menu screen  10  comprises a first Speak button  23  and second Speak button  27 . Keyboard  11  on Normal Keyboard Menu screen  10  is operatively arranged to allow a user to enter alphanumeric characters by selecting the keyboard keys having alphanumeric characters via an input device. Alphanumeric characters are displayed on View screen  19  upon receiving an input signal from the user via an input device. The alphanumeric characters may comprise words, phrases, and sentences. Upon a user beginning to input words, phrases, or sentences, the computer will predictively complete the word, phrase, or sentence. After a user has inputted alphanumeric characters via an input device, and the desired word, phrase or sentence is completed via the user and/or the predictive capability of the device, Speak button  23  or  27  may be selected. Upon selecting speak button  23  or  27 , the text-to-speech converter delivers the converted text-to-speech through the onboard speakers. It should be apparent to those having ordinary skill in the art that other means may be used to generate an input signal. For example, a synthesized voice may be operatively arranged to whisper menu selections and other screen information to a user, wherein a user triggers an input device when the user hears the desired selection. Further, the system may comprise an auto-scan mechanism wherein the menu choices are automatically scanned and a user selects a desired choice via an input device when the desired choice is indicated.  
         [0023]     Matching Sentences window  22  comprises the predictive capability described in detail supra. Once a user begins to enter a sentence into View screen  19 , a list of sentences that match the entered information appears in Matching Sentences window  22 . A user may then select an entire sentence, rather than entering the sentence verbatim into View screen  19  via Keyboard  11 . The phrase completion is based on a dynamic prediction. By “dynamic” it is meant that the statistics are calculated each time a word, phrase, or sentence is entered and thus, the device becomes easier to use over time, since it adapts to a user&#39;s habits and patterns. An individual having a speech impairment is therefore, able to communicate with others quicker and more efficiently.  
         [0024]     In the preferred embodiment, Sentence Beginners window  24  provides a list of predetermined beginning sentence phrases. Upon selecting a provided beginning sentence phrase, the phrase will appear in View screen  19 . Matching Words window  25  comprises the predictive capability described in detail supra. Once a user begins to enter a word into View screen  19 , a list of predicted words appears in window  25 . A user can select a word in window  25  such that the word appears in View screen  19 . In window  25 , words are automatically populated according to the predictive capability of the present invention, wherein a database is used to tag and store a user&#39;s preferred words according to the frequency used, most recent used, time of day used, day of week used, and time of year used. Finally, Word Enders window  26  provides a list of predetermined word endings. Upon selecting a provided word ending, the word ending will appear in View screen  19 . It should be readily apparent that Normal Keyboard Menu screen  10  may comprise a plurality of prediction windows and these modifications are intended to be within the spirit and scope of the invention as claimed. In some aspects, a Global Positioning System (GPS) is incorporated such that words, phrases, and sentences are also predicted according to location. Therefore, a context-awareness is provided and may be based on a plurality of factors including: location, frequency of use, most recent used, time of day used, day of week used, and time of year used. It should be appreciated that words, sentences, and phrases may also be completely entered by a user or pre-programmed into the database via an applications programming interface. The applications programming interface may be used to alter and/or add statistics, database characteristics, login functions, and to provide other basic capabilities. Further, menu target size, menu selections, colors, volume, and parameters affecting content prediction and phrase completion may be configurable. The database further comprises internal tracking means wherein a history of relevant parameters such as the number of corrected or rejected phrases is tracked to improve the long-term evolution of the device.  
         [0025]     Help button  14  of Normal Keyboard Menu screen  10  is an important feature of the present invention. Upon selecting Help button  14 , a voice will immediately speak a predetermined message, wherein others are alerted by the device&#39;s voice that the user needs urgent assistance. In a preferred embodiment, the volume of the voice speaking the predetermined message will be increased when Help button  14  is selected. Commercial text-to-speech engines that are well known in the art may be used as the synthesized voice. For example, a plurality of AT&amp;T&#39;s Natural Voices™ may be provided such that the user may choose among a variety of voices. It should be appreciated that a pre-recorded voice of the user or another individual may be used as well.  
         [0026]     The system further comprises an emotion-component wherein the voice&#39;s intonation may be changed according to the user&#39;s current emotions. For example, a user may select an angry mode, shout mode, polite mode, whisper mode, or normal mode such that the inflection of the device&#39;s speech is altered accordingly. It should be appreciated that a predictive emotion feature is contemplated wherein a synthesized voice speaks in a mode determined according to the user&#39;s biometric measurements and via algorithmic filtering. For example, if a user is angry and enters information to be spoken, the synthesized voice would have an increased volume and change of intonation as a result of the user&#39;s change in biometric measurements.  
         [0027]     To remove or delete entered alphanumeric characters from View screen  19 , Clear button  20  may be selected. Upon selecting Keyboard button  18 , Special Keyboard for David screen  54  is launched as described in detail infra and as shown in  FIG. 5 . Similarly, upon selecting Notepad  13 , Notepad screen  56  appears as described in detail infra and as shown in  FIG. 6 . Upon selecting History button  15 , a chronological list of words, phrases, and sentences that have been previously converted from text-to-speech is provided. Upon selecting Swap Keyboard button  12 , Keyboard  11  is switched from a standard keyboard layout as shown in  FIG. 1 , to a specialized keyboard designed according to a user&#39;s preferences. The specialized keyboard may appear similar to Keyboard  55  shown in  FIG. 5 .  
         [0028]      FIG. 2  is a photograph of Categories Menu screen  28  of the present invention. Categories Menu screen  28  is launched upon selecting Categories button  16  of Normal Keyboard Menu screen  10  of  FIG. 1 . Categories Menu screen  28  broadly comprises a plurality of category buttons. Each category button further comprises a category title and corresponding category icon. For example, Breakfast button  29  and Greetings button  30  have category titles “Breakfast” and “Greetings,” respectively, and a corresponding icon. Notepad button  13 , Help button  14 , History button  15 , Categories button  16 , and Keyboard button  18  of the screen of  FIG. 1  also appear on Categories Menu screen  28  of  FIG. 2 . Having Keyboard button  18 , Categories button  16 , History button  15 , and Help button  14  appear in the same position on every screen allows a user to easily locate and launch a screen regardless of the current screen being viewed. It should be apparent that although graphics are used to illustrate the category choices, photographs or other picture types may be used to illustrate the categories. Further, it should be appreciated that categories other than those provided in Categories Menu screen  28  are contemplated and these modifications are intended to be within the spirit and scope of the invention as claimed.  
         [0029]      FIG. 3  is a photograph of Breakfast Menu screen  29 , which is accessed by selecting Breakfast button  29  of Categories Menu screen  28  of  FIG. 2 . Breakfast Menu screen  29  broadly comprises windows  30 ,  31 ,  32 ,  33 , and  34  with corresponding Category Picture buttons  35 ,  36 ,  38 ,  39 , and  40 . Windows  30 ,  31 ,  32 ,  33 , and  34  comprise a plurality of preprogrammed words, phrases, and sentences, which are categorized and displayed by topic to improve a user&#39;s ability to access information according to the context of a conversation. For example, window  30  comprises a list of sentences related to a user&#39;s general breakfast needs such as the sentence “I don&#39;t want breakfast.” Window  31  comprises a list of sentences related to breakfast food preferences such as “I want scrambled eggs.” Windows  32 ,  33 , and  34  comprise a list of sentences related to beverage preferences, dairy product preferences, and special preferences, respectively. Upon selecting a sentence provided in window  30 ,  31 ,  32 ,  33 , or  34 , the sentence will appear in View screen  19 . The selected word, phrase, or sentence appearing in View screen  19  will be converted into speech when Speak button  23  is selected. The corresponding Picture buttons  35 ,  36 ,  38 ,  39 , and  40  help a user quickly identify the desired categories. This feature also allows an illiterate individual to use the device. For example, a user may select a relevant icon, listen to whispered sentence selections, and then select the desired sentence via an input device when the desired sentence is heard. Categories buttons  41  are provided so that other categories rnay be easily accessed directly from Breakfast Menu screen  29 .  
         [0030]      FIG. 4 a  photograph of Greetings screen  42 , which is accessed by selecting Greetings button  30  of Categories Menu screen  28  of  FIG. 2 . Greetings screen  42  is also accessible by selecting the Greetings button of the Categories buttons  41  shown in  FIG. 3 . Screen  42  displays windows and corresponding pictures that are relevant to the Greetings category. Therefore, windows  43 ,  44 ,  45 ,  46 , and  48  list sentences related to Greetings, Responses, Requests, Commands, and Niceties, respectively, with respective Picture buttons  49 ,  50 ,  51 ,  52 , and  53  displayed above. It should be apparent that each categories screen may have a plurality of category windows having preprogrammed responses, which are presented to a user in a context-relevant manner, and these modifications are intended to be within the spirit and scope of the invention as claimed.  
         [0031]      FIG. 5  is a photograph of Special Keyboard for David screen  54 , which is accessed by selecting Keyboard button  18  of Categories Menu screen  28  of  FIG. 2 . However, Keyboard button  18  appears in the same position on every screen allowing a user to easily locate and launch screen  54  regardless of the screen being viewed. Special Keyboard  55  allows a user to enter phrases and sentences that are not preconfigured into the system, thereby giving a user the flexibility needed in conversation. Special Keyboard  55  further enhances communication capabilities by providing a user with a quick and efficient keyboard layout. For example, the distance that the user has to move an input device to make a selection on screen  54  is minimized. Further, Special Keyboard  55  comprises numbers, letters, words, and word parts arranged according to a user&#39;s preferences. Further increasing the efficiency of Special Keyboard  55  is a color-coding feature, wherein different colors are used to indicate different keyboard areas. As in the photographs of  FIGS. 1, 3 , and  4 , entered words appear in View screen  19  and are spoken by the synthesized voice when Speak button  23  or  27  is selected. Also, as illustrated on Normal Keyboard Menu screen  10  of  FIG. 1 , Swap Keyboard buttons  12  are provided on Special Keyboard for David  54  screen so that a user may easily switch keyboard layouts if desired. Swap Keyboard buttons  12  provide the user a choice of switching between a standard keyboard layout as shown in  FIG. 1  and Special Keyboard  55 .  
         [0032]      FIG. 6  is a photograph of Notepad screen  56  of the present invention, which is accessed by selecting Notepad button  13  of  FIGS. 1-5 . Notepad screen  56  allows a user to pre-enter and store words, phrases, and sentences prior to a conversation, and is another novel feature of the invention. A user may pre-enter information into large View screen  58  and save the information for later use. When a user would like the device to speak the pre-entered conversation, the user may access Notepad screen  56 , select the pre-entered information, and then select Speak button  27 . In addition to Help button  14 , History button  15 , Categories button  16 , and Keyboard button  18  appear on Notepad screen  56 . Back button  59  is provided which brings a user to the previous screen.  
         [0033]     Thus, it is seen that the objects of the invention are efficiently obtained, although modifications and changes to the invention may be readily imagined by those having ordinary skill in the art, and these changes and modifications are intended to be within the scope of the claims.

Technology Category: 3