Abstract:
A transcription method uses a computerized process to prompt a student to produce at least one phoneme orally. Next a correct and at least one incorrect production of the phoneme are displayed. The therapist selects from among the displayed productions based upon the student-produced phoneme. The system includes a processor and display to prompt a student to produce at least one phoneme orally, display a correct and at least one incorrect production of the phoneme. The therapist then uses an input device in signal communication with the processor to select from among the displayed correct and incorrect productions based upon the student-produced phoneme, thus obviating the need for the therapist to enter the incorrect production symbol by symbol, unless it is desired to do so, or unless the actual production is not found among the displayed production selections.

Description:
CROSS-REFERENCE TO RELATED APPLICATION 
     This application is a continuation-in-part of and claims priority to application Ser. No. 09/769,776, entitled “Speech Analysis and Therapy System and Method,” filed Jan. 25, 2001, and application Ser. No. 09/770,093, entitled “Speech Therapy System and Method,” filed Jan. 25, 2001, which are commonly owned with the instant application and which are incorporated herein by reference. 
    
    
     BACKGROUND OF THE INVENTION 
     1. Field of the Invention 
     The present invention relates to systems and methods for analyzing and remediating speech pathologies, and, more particularly, to such systems and methods that are computer-based. 
     2. Description of Related Art 
     Articulation and phonology disorders are the most common of the speech and language disorders. The prevalence of this disorder is, at the time of writing, approximately 10% of the school-age population. In addressing a perceived articulation issue in a student, speech/language pathologists have in the past used an initial test based upon a series of cards. Each card contains a picture and a word, and the student is asked to pronounce the word associated with the card. The pathologist then determines whether the student&#39;s pronunciation is “right” or “wrong.” It may be recognized that such a system can be cumbersome, owing to the cards&#39; having to be placed in a desired order and sorted manually. 
     An intervention system designed to automate this process, Picture Gallery I, was presented by the owner of the current application. In this system pictures and/or words stored in a database could be sorted using a desired criterion such as a particular phoneme and presented to the student under software control for facilitating the acquisition or remediation of speech or language skills. No analysis or scoring is performed; rather, the product is intended for use by one or more students, either alone or in concert with a pathologist/teacher. 
     A previously known method of diagnosing articulation or phonology disorders included a “pencil and paper” test wherein a student is asked to speak a word. The therapist grades the word subjectively, based upon the therapist&#39;s ear and the local standards. 
     Other systems known in the art that address speech/language analysis and therapy methodologies includes those of Neuhaus (U.S. Pat. No. 6,113,393), Parry et al. (U.S. Pat. No. 6,077,085), UCSF and Rutgers (U.S. Pat. Nos. 5,813,862 and 6,071,123), Neumeyer et al. (U.S. Pat. No. 6,055,498), Jenkins et al. (U.S. Pat. Nos. 5,927,988 and 6,019,607), Siegel (U.S. Pat. No. 6,009,397), Beard et al. (U.S. Pat. No. 5,857,173), Aaron et al. (U.S. Pat. No. 5,832,441), Russell et al. (U.S. Pat. Nos. 5,679,001 and 5,791,904), Rothenberg (U.S. Pat. No. 5,717,828), Wen (U.S. Pat. No. 5,562,453), Ezawa et al. (U.S. Pat. No. 4,969,194), Sturner et al. (U.S. Pat. No. 5,303,327), Shpiro (U.S. Pat. No. 5,766,015), and Siegel (U.S. Pat. No. 6,148,286). Commercial software products in the field of articulation, phonology, or speech sound production include SpeechViewer, Interactive System for Phonological Analysis, Speech Master, Visi-pitch, and Computerized Profiling. Commercial print products include the Goldman-Fristoe Test of Articulation (American Guidance Service), Khan-Lewis Test of Phonology (American Guidance Service), Photo Articulation Test (Pro-Ed), and Fisher-Logeman Test of Articulation (Pro-Ed). 
     SUMMARY OF THE INVENTION 
     It is therefore an object of the present invention to provide a system and method for eliciting a desired sound from a user. 
     It is a further object to provide such a system and method adapted to generate a report. 
     It is another object to provide a system and method for testing a user&#39;s articulation. 
     It is an additional object to provide such a system and method that is adapted to analyze a group of problematic sounds. 
     It is also an object to provide such a system and method that recommends a therapeutic program responsive to the analysis. 
     It is yet a further object to provide such a system and method that includes a prescreening feature. 
     It is yet another object to provide a system and method for facilitating a therapist to transcribe speech of a student/client. 
     These and other objects are achieved by the present invention, a first aspect of which comprises a method and system for providing speech therapy. The method comprises the steps of selecting a problem speech sound and searching a database that comprises a plurality of records. Each record comprises a picture and a word associated with the word. 
     Next a set of records is automatically generated from the plurality of records. Each record contains a word specific to the problem speech&#39;s sound. The set of records is next automatically presented to a user sequentially on a display device, and the user is prompted to pronounce the displayed word. Finally, the pronunciation of each word is scored. 
     The system of the first aspect of the present invention comprises a processor, an input device in communication with the processor having means for selecting a problem speech sound, and a display device in communication with the processor. The database as described above is resident on the processor, as are software means. The software is adapted to automatically generate a set of records from the plurality of records, with each record containing a word specific to the problem speech sound. The software is also adapted to automatically present at least a portion of each record in the set of records to a user sequentially on a display device; the set of records to a user sequentially on the display device and to prompt the user to pronounce the displayed word. Finally, the software is adapted to receive via the input device a score for the pronunciation of each word. 
     Another aspect of the present invention is a system and method for analyzing a speech problem by performing a test of articulation, phonology, and sound features that is administered and analyzed with the use of an electronic processor. This method comprises the steps of presenting to a student/user a symbol representative of a word and prompting the user to pronounce the word represented by the symbol into a microphone in signal communication with a processor. Next the therapist enters a phonetic representation of the user pronunciation into the processor. It is then automatically determined whether an error exists in the user pronunciation. If an error exists, the error is automatically categorized. 
     In an alternate embodiment of the method, the therapist enters the phonetic representation of the user pronunciation into an input and storage device that is not in signal communication with the processor. At a later time the phonetic representation is downloaded into the processor, whereupon the automatic determining and categorizing steps proceed. 
     The system of the second aspect of the invention evaluates an articulation disorder. The system comprises a processor and an output device and an input device, each in signal communication with the processor. 
     Software installable on the processor is adapted to present on the output device, typically a display device, although this is not intended as a limitation, a symbol representative of a word. The software then is adapted to prompt a user via the output device to pronounce the word represented by the symbol and to receive from the therapist via the input device a phonetic representation of the user&#39;s pronunciation. The software automatically determines whether an error exists in the user pronunciation, and, if an error exists, automatically categorizes the error. 
     In the alternate embodiment the system comprises a processor and an output device and a user input device, each in signal communication with the processor. The system further comprises an operator input and storage device that is not in signal communication with the processor, but is connectable thereto for downloading operator-entered data thereinto, the data comprising the phonetic representation. 
     The software then receives downloaded data from the operator input and storage device the phonetic representation of the user&#39;s pronunciation. The software automatically determines whether an error exists in the user pronunciation, and, if an error exists, automatically categorizes the error. 
     The system and method of this second feature of the invention may be adapted for presentation of a single word, a plurality of words having a predetermined feature desired to be tested, a pretest for screening for potential articulation disorders, and an analysis of connected speech with the use of a moving picture to elicit a narrative from the student. 
     An additional aspect of the present invention is directed to the transcription of a student&#39;s speech by the therapist using a computerized process. This method comprises the steps of prompting the student to produce at least one phoneme orally. Next a correct production of the at least one phoneme is displayed to the therapist, as well as at least one incorrect production of the at least one phoneme. The therapist is then permitted to select from among the displayed correct and incorrect productions based upon the student-produced at least one phoneme. 
     The system related to this aspect of the invention comprises a processor and display means in signal communication with the processor. The display means are for prompting a student to produce at least one phoneme orally, displaying a correct production of the at least one phoneme to a therapist, and displaying at least one incorrect production of the at least one phoneme to the therapist. The therapist then uses input means in signal communication with the processor to select from among the displayed correct and incorrect productions based upon the student-produced at least one phoneme, thus obviating the need for the therapist to enter the incorrect production symbol by symbol, unless it is desired to do so, or unless the actual production is not found among the displayed production selections. 
     The features that characterize the invention, both as to organization and method of operation, together with further objects and advantages thereof, will be better understood from the following description used in conjunction with the accompanying drawing. It is to be expressly understood that the drawing is for the purpose of illustration and description and is not intended as a definition of the limits of the invention. These and other objects attained, and advantages offered, by the present invention will become more fully apparent as the description that now follows is read in conjunction with the accompanying drawing. 
    
    
     BRIEF DESCRIPTION OF THE DRAWINGS 
     FIGS. 1A,  1 B is a flow chart for an exemplary embodiment of the speech therapy method of the invention. 
     FIG. 2 is a schematic diagram of the speech therapy and analysis system. 
     FIGS. 3A,  3 B is a flow chart for an exemplary embodiment of the speech analysis method of the invention. 
     FIG. 4 is a section of a flow chart for another embodiment of the speech analysis method of the invention. 
     FIG. 5 is a schematic diagram of an alternate embodiment of the speech analysis system. 
     FIGS. 6A,  6 B is a flow chart for an additional embodiment of the speech analysis method of the invention. 
     FIG. 7 is an exemplary phonemic profile or individualized phonological evaluation screen. 
     FIG. 8 is an exemplary basic IPA production transcription screen. 
     FIG. 9 is an exemplary parent letter report. 
     FIG. 10 is an exemplary student production report option selection screen. 
     FIGS. 11A-11E is an exemplary level 1 treatment suggestion report. 
     FIGS. 12A-12E is an exemplary level 2 treatment suggestion report. 
     FIGS. 13A,  13 B is an exemplary level 3 treatment suggestion report. 
     FIG. 14 is an exemplary level 4 treatment suggestion report. 
     FIG. 15 is an exemplary connected speech sample transcription screen. 
    
    
     DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS 
     A description of the preferred embodiments of the present invention will now be presented with reference to FIGS. 1A-15. 
     A flow chart of an exemplary embodiment of the automated speech therapy/intervention method is given in FIGS. 1A,  1 B, and a schematic of the system in FIG.  2 . The system and method are also contemplated for use in the acquisition of a language skill as well as in a remediation setting. There are two versions of the system and method: In the “professional” version  10  of the invention (block  100 ), typically two people who will be referred to as “therapist”  11  and “student”  12  are present, although this is not intended as a limitation. This version is contemplated for use in such settings  32  as a hospital, clinic, rehabilitation center, school, or private facility. In the “personal” version  40  of the invention, the “student”  12  may be working alone, or in the presence of a nonprofessional such as a parent. The therapist  11  may be, for example, a speech therapist or a teacher; the student  12  may be a user who is learning a second language or a school attendee who is being tested for, or who is already known to have, an articulation problem or phonological disorder. 
     The method comprises the steps of providing access to an electronic database that includes a plurality of records (block  101 ). Each record comprises a word, a picture representative of the word, and a recommended pronunciation of the word. In an alternate embodiment, the record may also include a digitized video clip to represent motion or a verb to impart a concept of action. In another embodiment the record may further include a digitized sound that is associated with the word. For example, the record for the word dog might contain a picture of a dog, a video clip of a dog running, and/or a barking sound. It is believed that such multiple stimuli appeal to a multiplicity of cognitive areas, thereby optimizing the student&#39;s improvement. 
     Each record may further contain data useful for performing sorting functions, such as at least one category and/or concept. An exemplary set of categories comprises: animals, art, babies, celebrations, global images, environment, family, food, garden, health and exercise, home, leisure, medical, money, music, pets, play, school, shopping, signs/symbols, sports, technical, vacations, and work. An exemplary set of concepts comprises: activities, objects, places, people, ideas, and events. The record also typically comprises a vocabulary level associated with the word and a length of the word. 
     The method next comprises the step of inputting or accessing previously input demographic information for the student (block  102 ). Then a problem speech sound that is desired to be improved upon is selected that is known from a prior diagnosis (block  103 ). The problem speech sound may be selected from a group consisting of a phoneme and a “feature.” The feature comprises at least one of a place, a manner, and a voicing characteristic. Searching on a feature yields matches in all positions of words. The database is electronically searched (block  106 ) for records containing words that include the problem speech sound to generate a set of records. A filter may be applied if desired (block  104 ) to further limit the set (block  105 ), including selecting a category or concept, using the demographic information to limit the set, such as eliminating words that are intended for students over 7 years of age for a 5-year-old student, setting a desired vocabulary level, or selecting a word length. 
     If desired (block  107 ), the set of records may also be sorted (block  108 ) in various ways to produce a desired sequence, including, but not limited to, putting the words in alphabetical order, random order, or some other chosen sequence. In a preferred embodiment, all the words in the database contain at least one of the letters “r,” “l” and “s,” since these are known to present a problem most frequently. 
     For a professional therapy session, a decision may be made whether to present the set of records or store/transmit them (block  109 ). If the former, the set of records is next presented sequentially to the student in the predetermined sequence on a display device (block  111 ), and the student is prompted to pronounce the word (block  112 ). The display style may be selected (block  110 ) from a word only, a picture only, or a word plus a picture. 
     If the student can read, he or she can use the displayed word to form a pronunciation; if the student cannot yet read, or cannot read the currently presented language, the picture will also aid in acquisition of reading skills as well as pronunciation. 
     In the professional setting, the therapist scores the student&#39;s pronunciation (block  113 ) by inputting, for example, “correct,” “incorrect,” “skip,” or “re-present,” which will record an indication to re-present the record at a later time, such as after all the other items in the set have been presented. The student or therapist can also elect (block  114 ) to hear the word pronounced (block  115 ) in a recommended manner by making an appropriate selection on an input device. 
     The scores are received by the system, and an aggregate score is calculated (block  116 ) for the problem speech sound. The database also comprises a historical record of all sessions for each of the students, and the database is then accessed to store the current score thereinto (block  117 ). The therapist may choose to calculate a historical change (block  118 ) from previously saved scores to provide an indication of the student&#39;s progress. Such scores may also be used to calculate statistics (block  119 ) for a group of students, using, for example, a demographic filter. 
     The “personal version” of the system and method does not accept scoring, nor is there a database from which sets of records may be created. Rather, the professional version is adapted to download a selected set of records onto a storage medium, such as a diskette, or to transmit the set of records to a remote site (block  109 ). Such a remote site may comprise, but is not intended to be limited to, a room remote from the main processor accessible via intranet, or a different building accessible via internet. This version then enables the student to perform (block  120 ) the steps in blocks  110 - 112  and  115  as desired on his or her own. 
     The system  10 , as schematically illustrated in FIG. 2, comprises a processor  14 , on which are resident the software package  15  of the present invention adapted to perform the functions as outlined above and a database  16  comprising the plurality of records  17  and demographic and historical data on the users  12 . An input device is in communication with the processor  14  that has means for selecting a problem speech sound. Such means may comprise any of the devices known in the art such as a keyboard  18  or pointing device such as a mouse  19  or touch screen. A display device such as a display screen  20  is also in communication with the processor  14 . 
     Optional elements that are also in communication with the processor  14  may include a microphone  21  and a speaker  22 , both under processor  14  control, as well as means for performing analog-to-digital  23  and digital-to-analog  24  conversions. The system  10  also has means for transferring records from the database to a storage medium such as a disk drive  25 , under control of the software  15 , or to a remote site such as another location  26  via a modem  27  over the internet  28  or such as another room  29  at the same location via an intranet  30 . A printer  31  under processor control may also be provided for furnishing a hard copy of any portion of the session as desired. 
     A secondary system  40  for use of the personal version of the invention at the remote location  26 ,  29  comprises a processor  41 , input device  42  and display device  43  in communication with the processor  41 , and either or both of a modem  44  for receiving a set of records and a storage device reader  45  for reading a stored set of records. The software package  46  for this version is adapted to read the records, present them to the student  12  sequentially, and prompt the student  12  to pronounce the word associated with the record. 
     A flow chart of an exemplary embodiment of the automated speech therapy/intervention method is given in FIGS. 3A,  3 B. The schematic of the system is substantially the same as that in FIG.  2 . The method comprises the steps of selecting the type of evaluation desired to be performed (block  501 ): screening, single word analysis, “deep” test, or connected speech analysis. The screening, or pre-evaluation, comprises the steps of presenting to a user a symbol representative of a word (block  502 ) and prompting the user to pronounce the word represented by the symbol into a microphone in signal communication with a processor (block  503 ). The symbol presentation may comprise, for example, a picture on a display screen, although this is not intended as a limitation. The therapist then enters a phonetic representation of the user pronunciation into the processor (block  504 ). 
     In an alternate embodiment of the method, the altered portion of which is illustrated in FIG. 4, the therapist enters the phonetic representation of the user pronunciation into a separate operator input and storage device  47 , such as, but not intended to be limited to, a personal data assistant (block  520 ). At a later time, the user pronunciation data are downloaded into the processor (block  521 ) to complete the steps of the method. 
     A schematic of the system (FIG. 5) illustrates the addition of the operator input and storage device  47 , which is connectable to the system  10  when desired for downloading data into the processor  14  that has been entered thereinto by the therapist  11 . 
     The advantages of this embodiment include the user and the operator being able to use separate pieces of hardware, thereby eliminating physical restraints imposed by attempting to share equipment. Further, during the session the user cannot view the operator&#39;s scoring information, which may inhibit the user. In addition, the operator&#39;s hardware may retain data for downloading into more than one processor if desired for subsequent collection and analysis. 
     In both embodiments, the software installed upon the processor then automatically determines whether an error exists in the user pronunciation (block  506 ). The determination may additionally include the application of a dialectical filter (block  505 ) that is adapted to discriminate between that which is deemed to be a true error and a predetermined normal dialect word pronunciation. If an error exists, the software automatically categorizes the error (block  507 ). An error may be, for example, a substitution, a mispronunciation, or an omission. These steps are repeated a predetermined number of times n, for example, 20 times (block  510 ). 
     It may then be desired to perform the “deep test,” which may be performed with the knowledge gained from a pre-evaluation as above or de novo. If the pre-evaluation has been performed, the software automatically generates a set of symbols, wherein each symbol is representative of a word containing at least one of the errors determined in the pre-evaluation. Then the steps as above are performed using the generated set of symbols, and an evaluation is made of articulation errors for the whole set. 
     If a single word is desired to be analyzed for, the steps in blocks  502 - 509  are performed once for the desired word. 
     Once a word has been pronounced and the phonetic representation entered into the processor, the therapist may decide to display a frequency spectrum of the user&#39;s pronunciation (block  508 ). If desired, a sample of a correct pronunciation of the word may be broadcast via a speaker in signal communication with the processor (block  509 ). 
     When a plurality of words have been tested, the evaluating step also comprises automatically recognizing an underlying commonality by correlating the errors detected. This pattern recognition permits the software to achieve an overarching diagnosis of a problem speech sound (block  511 ). 
     Following the error categorization, if desired, a report can be issued detailing the user&#39;s error(s) (block  512 ). Additionally, the error may be saved in a database that is accessible by the processor (block  513 ). If a previous entry for this user already exists, which is determined by a search, the error found in the present test may be compared with an error previously found, and a change over time determined for that user (block  514 ), to note whether an improvement has occurred. Again, if desired, a report may be issued (block  515 ) as to the change determined. 
     An additional feature of this invention is the ability, once a categorization has been made of an error, of recommending a therapeutic program to address the error (block  516 ). Such a recommendation formulation may comprise, for example, creating a set of records as detailed above in FIGS. 1A-2. 
     If connected speech analysis is desired to be performed, the “symbol” comprises a motion picture representative of an action, and the user is prompted to provide a narration on the action into a microphone in signal communication with a processor. The therapist then enters a phonetic representation of the user&#39;s pronunciation of the narration into the processor. Software resident in the processor automatically determines whether an error exists in the user pronunciation, and, if an error exists, automatically categorizes the error. 
     Another aspect of the present invention relates to a system and method for transcribing student-produced speech by a therapist (FIGS.  6 A- 15 ), for analyzing the transcribed speech, and for producing a report and recommendations based upon the analysis. The steps of the method are illustrated in flow-chart form in FIGS. 6A,  6 B, and exemplary screens, letters, and reports in FIGS. 7-15. The system of the invention is substantially as illustrated schematically in FIG. 2 within the “professional site”  32 . 
     The method of the present invention includes the steps of entering student and therapist information (block  601 ), such as demographic information. The therapist  11  is then permitted to choose (block  602 ) between administering a “phonemic profile” (block  603 ) or a “connected speech sample” (block  604 ), and also whether or not to record the student&#39;s production. In either case, the therapist  11  may select between basic English International Phonetic Alphabet (IPA) or full IPA. 
     If the phonemic profile selection is made, a stimulus is presented to the student (block  605 ), such as by displaying a picture on the screen  20  to elicit a particular sound, which may comprise one or more phonemes. For example, a picture of a cat would elicit the student to say “cat.” 
     For each stimulus, the correct target word (e.g., “cat”) and predicted incorrect productions (e.g., “tat”) are displayed on the screen  20  to the therapist  11  (block  606 ) in, for example, IPA format. The therapist  11  is then permitted, if a match occurs (block  607 ), to select from among the displayed options based upon the student&#39;s production (block  608 ) or to enter the student&#39;s production in IPA format (block  609 ). The selection of block  608  is made, for example, by a “point and click” method using the mouse  19  on a screen such as FIG. 7; the production entering of block  609  may also be made by a “point and click” method using the mouse  19  on a transcribing screen such as in FIG.  8 . 
     Once the phonemic profile is complete (block  610 ), the software package  15  performs an automatic analysis for the student (block  611 ), displays the results of the analysis on the screen  20  or prints the analysis results on the printer  31  (block  612 ), applies a filter such as an age and/or a dialect filter (block  613 ), and displays the results of the analysis with applied filter(s) on the screen  20  or prints the analysis results on the printer  31  if one or more filters were applied (block  614 ). Then the analysis is used to prepare a narrative parent letter and/or report that includes problem sounds (FIG. 9) and recommendations for treatment (block  615 ). An example of available student production report selections is shown in FIG.  10 . Additional report selections include descriptions of student productions; word length, stress pattern, and word shape inventories; and consonant and vowel inventories. 
     If desired (block  616 ), the therapist  11  can proceed to an individualized phonological evaluation (IPE). The stimuli for this evaluation are determined based upon the results of the phoneme profile, and there are four levels of evaluation possible, as will be reflected in the treatment reports to be discussed in the following. For example, if the student pronounced “tat” for “cat,” words such as “can,” “call,” “cad,” or “cast” may be selected for presentation to the student  12 . Once again, stimuli, transcription, and analyses are performed analogous to blocks  605 - 611  (block  617 ), with the analysis based upon both the phonemic profile and the IPE, and a report, a letter, and treatment recommendations provided analogous to blocks  612 - 615  (block  618 ). Exemplary treatment suggestion reports for four levels of IPEs are shown in FIGS. 11A-11E,  12 A- 12 E,  13 A- 13 B, and  14 . 
     If the connected speech option was selected (block  604 ), a stimulus is presented to the student  12  (block  619 ), such as a video clip on the screen  20  or other external stimulus. The therapist  11  determines an intended target sentence (block  620 ) as the student&#39;s production is made. The therapist  11  enters the target production on the keyboard  18  in orthographic format (block  621 ; FIG.  15 ), and the system  15  converts it into IPA format (block  622 ). The student&#39;s production is defaulted to be the target production (block  623 ), and the therapist  11  edits the production fields in order to convert it into the actual student production (block  624 ). 
     Once the editing is complete, the production is analyzed (block  611 ), with a comparison being made between the target and actual productions. Reports are then displayed (block  612 ) on the student&#39;s production and the comparison. The remaining blocks are substantially the same as with the phonemic profile. 
     It may be appreciated by one skilled in the art that additional embodiments may be contemplated, including alternate forms of presentation of the symbols and sounds. 
     In the foregoing description, certain terms have been used for brevity, clarity, and understanding, but no unnecessary limitations are to be implied therefrom beyond the requirements of the prior art, because such words are used for description purposes herein and are intended to be broadly construed. Moreover, the embodiments of the apparatus illustrated and described herein are by way of example, and the scope of the invention is not limited to the exact details of construction. 
     Having now described the invention, the construction, the operation and use of preferred embodiment thereof, and the advantageous new and useful results obtained thereby, the new and useful constructions, and reasonable mechanical equivalents thereof obvious to those skilled in the art, are set forth in the appended claims.